I think you should read this howto: How to Install and Configure Piaware 5.0 SD card image - Quickstart Guide
This error when running rtl_test is always due to some program using it still running.
dump1090-fa and dump978-fa get restarted by piaware, so you need to stop that as well.
Also when you reboot the services obviously start up again.