Source packages?

Is there something like mock for .deb-based systems? If there is, it would be easy to produce clean debs for different systems and make a repo of all of them.

For the benefit of those too lazy to follow and read the link: mock creates a minimal system in a chroot, and uses it to build packages in a fully automated way. The chroot system does not need to be the same as its host system, so you can build clean fedora32 rpms on centos5 and vice versa (in fact you can also use it to build fedora and centos rpms on debian). The chroot is cleaned after each build, so there can be no unintended linking by Makefiles to stuff that was not foreseen by the build’s specfile. But mock is yum/dnf-based and 100% rpm-oriented.

pbuilder / cowbuilder is what we use. There are probably others.

This is indeed a big advantage of building within a clean environment. However the problem that abcd has is not due to an unclean environment - it’s because the dependent packages that were used to do the build on one distro variant are not available on another distro variant.

Just run following two bash commands, and they will do everything (i.e. clone source, install dependencies, build package & install the package). Isn’t copy-paste a bash command as easy as apt-get install?

Successfully tested on:

  • Debian 10.6 amd64
  • Ubuntu 20.4 amd64
  • Kali 2020 amd64

DUMP1090-FA 4.0

sudo bash -c "$(wget -O - https://raw.githubusercontent.com/abcd567a/piaware-ubuntu20-amd64/master/install-dump1090-fa.sh)"

PIAWARE 4.0

sudo bash -c "$(wget -O - https://raw.githubusercontent.com/abcd567a/piaware-ubuntu20-amd64/master/install-piaware.sh)"

1 Like

My dump1090-fa specfile says ‘BuildRequires: ncurses-devel’, FWIW.

There should be a premium for harmonising package names across distros (and some mild punishment for not doing so; quartering by horses comes to mind).

Just messing around in my VMWare and get the following- I initially was doing it manually as I always do, but figured I missed something silly, so went the script route - same error:

I have also tried building libusb and librtlsdr from source (as I always do) and got the same error on those attempts. FYI, latest Debian 10.6.0 amd64

Underlying libc and libusb are installed

@Nitr0

Strange!
I first tested the bash script 18 days ago on Debian 10.6, then again last week when I made a fresh install of Debian10.6, and it worked ok both times.

Ok, tonight I will make a fresh install of Debian 10.6 on Oracle VM, run the bash script, and let you know the outcome.

:innocent: am I allowed to enjoy this? I consider myself old but my workplace really wants to take everything into docker because it’s en vogue, whether the app is written as microservice or not. (Mostly not, of course.) Docker isn’t for sandboxing but a logical extension of chroot. I “docked” my then employer’s Java applications into multiple chroot in the early 2000s myself and saved them millions as well as the disgrace of running server-side Java on Windows. Then Sun (remember Scott McNealy?) wrapped chroot into Solaris 9 and tried to sell it as alternative virtualization before officially calling the technology “container” in Solaris 10.

More seriously, there are discussions about the existing docker image (I think for multi-feeder) being problematic, too. I will study USB issues more. (Never used anything from USB in VM.)

Good to know. Must say that VMware is superstable. They also have a “market place” for virtual appliances for people to find stuff. Does VirtualBox have one? (I toyed with VB for like half an hour years ago. When the image that I was interested in didn’t work, I gave up.)

I honestly can say that I haven’t touched Virtual Box since because it creamed the USB packets, so it wasn’t even in my wheelhouse anymore. Bear in mind, this was probably at least 4 or so years ago and I imagine VB has come a long ways just as VMWare has. Once I started using VMWare, I haven’t looked back due to the stability and ability to expand existing partitions, etc that VB didn’t ave at the time. Not bad-mouthing VB by any means - it’s a free application and did it’s job for the most part, just not effective for my needs. It may be perfect for others.

A virtual appliance running a (plug your favorite OS) VM is totally different from running a XXX VM yourself. The idea is to let the end user run the PiAware image (ARM) without touching any of the middle layer. You download this appliance, boot it up, and you have a station. No compiling. No apt-get. (Networking will not be Ethernet or WiFi as the original image expects. That will have to be worked out.) You do not have to know what happens in between the virtual box and the antenna if you don’t care to. If you care about the Raspbian, you can still touch that empty ssh file and get yourself a shell, while still be totally ignorant that a middle VM is actually hosting Raspbian.

Of course, this is easier said than done. Right now, it’s just a technical path to further lower barrier of entry. (I haven’t done my research. It is not out of realm of possibilities that someone have already developed a generic appliance that allows you to run an arbitrary Raspbian image that you downloaded from anywhere.)

Building librtlsdr from source is the cause of your problems. Now you have a librtlsdr installed that is not managed by or known to the Debian packaging system. When you later try to build a dump1090-fa package, the build produces binaries that are linked to that unmanaged library. (Note that it has linked to the copy in /usr/local/lib, not the copy you ran ldd on). The Debian package builder then rightfully complains with that “no dependency information” error and refuses to produce a package that depends on a library that’s not visible to the package system (such a package would be unusable on any other system)

If you’re trying to emulate a full Pi for some reason, then probably your main stumbling block is going to be how to emulate the VideoCore.

I don’t get why you’d do this if you wanted to make a virtual appliance, though. If you have the knowledge to build such a virtual appliance image, you surely have the knowledge to build piaware/dump1090 from source and produce an x86 virtual appliance image. The only reason I could see for sticking with ARM binaries on an x86 host would be if you had some closed source ARM binaries that you could not obtain x86 equivalents for. That is not the case here.

Video core is an excellent pointer that I hadn’t thought about. Thank you! And the point about closed-source ARM binary is totally valid. In this case, the appliance could be a simple Debian VM with zenonp’s packages preinstalled.

I’m not aware of MikeNye’s docker images causing USB issues which is why i’d prefer them over a VM solution.

Or you know the person publishing that Devian VM image could take 5 minutes and compile the required programs.
Because if you’re publishing a VM image … you can certainly compile with a couple of commands noted on the piaware github page.
And dump1090-fa is really easy to compile.
Well i say compile while i mean build the debian package.

Usually I’d agree with you, but this is my flow for every system I install:

## OPTIONAL:  PRE-INSTALL PRE_REQS TO KEEP DUMP1090-FA BUILD FROM BITCHING ##
sudo apt install -y libusb-1.0-0-dev librtlsdr-dev
sudo apt-mark hold libusb-1.0-0-dev librtlsdr-dev

## Build and install -y libusb-1.0-0-dev FROM SOURCE ##
sudo apt install -y libc6 libudev-dev
git clone https://github.com/libusb/libusb.git
cd libusb
./autogen.sh
make -j4
sudo make install
sudo ldconfig
cd

#####################################
git clone https://github.com/osmocom/rtl-sdr.git
cd rtl-sdr
mkdir build && cd $_
CFLAGS="-march=native" cmake ../ -DINSTALL_UDEV_RULES=ON -DDETACH_KERNEL_DRIVER=ON
make -j4
sudo make install
sudo cp ../rtl-sdr.rules /etc/udev/rules.d/
sudo ldconfig
cd

Is there a better method? I install package, then I hold, build install source. Packages should be linked in this scenario no?..

If you can spare insight on how the libs don’t get linked in this instance, I’m all eyes. Would love to learn a more proper way of handling it.

This make install just essentially copies files into /usr/local. Debian’s package management system knows nothing about what those files are or what package they should belong to.

(if “dpkg -S /usr/local/lib/whatever-library-got-linked-to.so” can’t find a corresponding package, then you’ll run into problems. dpkg-shlibdeps looks for binaries, runs ldd on them, finds the owning packages for the dependent libraries, and add dependencies on those packages. If there’s no owning package, it yells at you)

If you wanted to rebuild librtlsdr and have it play nicely with the debian package management system, then you need to actually (re)build a debian package for librtlsdr and then install the package. The git repo does provide debian control files so it is probably just a case of dpkg-buildpackage -b (maybe after tweaking debian/rules as desired) and then dpkg -i on the resulting packages.

Debian 10.6.0 /Oracle VM / Windows 10

Total Time Taken By Above Bash Script = About 2 minutes (depends on speed of computer)

 

Total Time Taken By Above Bash Script = About 3 minutes (depends on speed of computer)

 

 

abcd@debian10:~$ sudo cat /var/log/piaware.log
... ... ...
Oct 24 11:32:08 debian10 piaware[614]: my feeder ID is xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Oct 24 11:32:08 debian10 piaware[614]: site statistics URL: https://flightaware.com/adsb/stats/user/abcd567#stats-114692
... ... ...
... ... ...
Oct 24 11:32:09 debian10 piaware[614]: mlat-client(641): fa-mlat-client 0.2.11 starting up
Oct 24 11:32:09 debian10 piaware[614]: mlat-client(641): Using UDP transport to 70.42.6.225 port 10293
Oct 24 11:32:09 debian10 piaware[614]: mlat-client(641): Listening for Beast-format results connection on port 30105
Oct 24 11:32:09 debian10 piaware[614]: mlat-client(641): Listening for Extended Basestation-format results connection on port 30106
... ... ...
Oct 24 11:37:40 debian10 piaware[614]: 168 msgs recv'd from dump1090-fa (168 in last 5m); 168 msgs sent to FlightAware

MLAT Working

image

 

 

Thanks @obj. Figured this is exactly what’s happening, but also thought the correct ‘hack’ was to install the libs from the management system, then hold, compile and install over the top. For some reason, I’ve never run into a problem with doing so before. Blind luck for a long time I guess, since the ownership would obviously be an issue as you pointed out.

@abcd567, I’ll give it another run today if I find the time. I probably should have just stayed in bed yesterday as it was one of those days where nothing worked right.

I used them. They worked flawlessly. On scripting I’d give you an A+. On teaching linux newbies to run bash -c "$(remote_unseen_and_unknown.sh)" as root, you merit an F-.

The result is a debian 10.6 edit: live USB image superseded, see below. That SDR dongle I ordered from the UK 10 days ago is still in the mail, so I haven’t been able to test. I’d love feedback from anyone with an x64 laptop and a spare dongle who would like to give it a go.

Notes:

  • Minimum size of USB stick is 16G. The unpacked image is only slightly smaller. It’s a full graphical system with KDE on writable ext4; no squashfs, no persistence overlays, it can be used just as any ordinary HDD-based system.
  • Only BIOS boot works. I’ll add EFI boot later.
  • The password for root and for user pi is “1090”. Do change them at the first opportunity.
  • The image doesn’t have a feeder-id, so it will create a new one as soon as it is booted. Please remember to delete the feed from FA’s servers when you’re done playing with the stick.
  • The system reports itself (image-type) as fabian64[*1]. That’s not recognised as a valid feeder type by FA’s adsb/stats/user web interface, but it can be used internally to track the uptake or lack thereof of the live-USB.

.

[*1] It just happened to make sense as a name for FA-debian. Later we might have fadora and fantos and, what do you know, even fandows, god forbid.

I will wait impatiently for fandows10. It is a fantastic idea. :slight_smile:

I would love to test “fabian”, but I have discarded my old Laptops. Now have only Desktop, which is EFI boot lloader.