Wolfi's bloghttps://blog.fossencdi.org/2017-04-17T20:20:00+02:00Should free software distributions host and provide non-free software?2017-04-17T20:20:00+02:002017-04-17T20:20:00+02:00Wolfgang Wiedmeyertag:blog.fossencdi.org,2017-04-17:/free_projects_host_nonfree.html<p class="first last">Some thoughts about the question if free software distributions should include non-free software and how this relates to Replicant and Debian. An attempt at a fitting analogy is made at the end.</p>
<p><a class="reference external" href="https://blog.fossencdi.org/why_free_software.html">I care a lot about free software</a>, not only for security and privacy reasons. Nowadays, I mainly contribute to <a class="reference external" href="http://www.replicant.us/">Replicant</a> to help advance free software. Replicant does not ship any non-free software and the website and wiki do not offer instructions to install non-free software. As a consequence, some functionalities are missing when installing Replicant because Replicant cannot yet provide free replacements for all parts that require non-free software, only for some.</p>
<p>Replicant's primary goal is to provide a fully free operating system that can be used on phones and tablets. Providing non-free software that makes the parts working that do not yet work with free software, would go against this primary goal. But one might argue that it makes sense to offer these non-free software pieces because some users really need the functionality that would be missing otherwise. And these users probably will not use Replicant if the functionalities they need are missing and they will use a different operating system that not only ships these non-free software pieces, but even more non-free software than the user would need to get the needed functionalities working. Wouldn't it be better if these users could still use Replicant instead of an operating system that has a lot more non-free software?</p>
<p>I think that there is no need for a free software distribution to make a compromise in this way and offer non-free software for critical functionalities while violating its primary goal at the same time, just to make it usable for more users. Someone will simply provide the non-free software and instructions to install it outside of the project. If there is a need for it, then this will happen in any case. This is not under the control of the project and it shouldn't be. Users can install non-free software on a free operating system if they want, but they shouldn't expect that the free software project is OK with it and helps with instructions or even forces the non-free software onto its users.</p>
<p>I recently thought again about this topic when working on the <a class="reference external" href="http://redmine.replicant.us/issues/1629">F-Droid issue</a> where we try to find a way to make the F-Droid client shipped with Replicant compliant with the <a class="reference external" href="https://www.gnu.org/distros/free-system-distribution-guidelines.en.html">GNU Free System Distribution Guidelines (GNU FSDG)</a>. Unfortunately, work like this sometimes gives the impression that Replicant only tries to remove non-free software and promotion of non-free software and doesn't really contribute any new free software. This couldn't be further from the truth as the main work Replicant developers are doing is to write free software to make hardware work that does not yet work with free software.</p>
<p>I'm a Debian user and I hope that the Debian project will someday take the same stance and remove all non-free software from its repositories. Debian already provides the non-free software in separate repositories and not in the main repository, but it still hosts these repositories and provides instructions to add them to install the non-free software. There is also the problem that some free software in the main repository advertises non-free software. For example, Firefox offers non-free addons alongside free ones. Of course, the Debian project can freely decide whether they provide non-free software or not and the decision should be respected. I'm just a user voicing his opinion.</p>
<p>I believe the following analogy captures pretty well how I feel about all of this:</p>
<p>You own a nice restaurant. You really care for your customers. You want them to eat tasty and healthy food and be happy about the service. They should feel welcome when entering your restaurant and feel great when leaving.</p>
<p>But there are some folks you just cannot convince to eat at your place. They do not want to wait so long for their order and the menu is too expensive for them. You know some of these people, some of them are friends of yours. So you want to accommodate them as well. And you can make a quick buck when they at least buy some drinks at your bar. Some time ago, you bought some space next to your restaurant. Now you decide to rent this space to a fast food chain. The people you could not convince to visit your place are now at least stopping by for a drink or two and maybe a snack when going to the fast food restaurant next door.</p>
<p>So you have now a little more customers and make some money with the rent to the fast food joint. But it turns out that the fast food restaurant is also competition as some of your core customer base comes less frequently and the general spirit of eating high quality and healthy food is not quite there anymore. So has renting the space next door to a fast food chain really helped your original goals? Probably not at all, maybe even the opposite. Yes, you still have more customers and more money overall, but at which cost?</p>
Problems getting a current kernel working on the Galaxy S3 and why merging is bad2017-02-02T17:32:00+01:002017-02-02T17:32:00+01:00Wolfgang Wiedmeyertag:blog.fossencdi.org,2017-02-02:/newer-kernel-galaxys3.html<p class="first last">Some time ago, I started to wonder if it would be possible to update the old Linux kernel for the GT-I9300 variant (international version) of the Galaxy S3 to a more recent version.</p>
<p>Some time ago, I started to wonder wheter it would be possible to update the old Linux kernel for the GT-I9300 variant (international version) of the Galaxy S3 to a more recent version. A newer kernel could not only bring new features and speed improvements, but also lots of security fixes that are not yet patched. Furthermore, porting a more recent Android version to the device would be much more easier. Making a newer kernel work with an older user space is much easier than making an old kernel work with a new user space. And isn't it the spirit of free software to be able to use the newest and shiny software on arbitrary hardware, regardless of its age?</p>
<p>But why Galaxy S3? Wouldn't it be better to take a more recent device that was already released with a newer kernel? I work on the Galaxy S3 because I use it as my daily driver and I have a spare device that I can use for development. Besides the Galaxy S2, it is still the best supported device by <a class="reference external" href="http://www.replicant.us/">Replicant</a> which was my original reason to start working with this phone. And especially in regards to the mainline kernel, it actually turns out to be a really good choice, but more about that in another post.</p>
<div class="section" id="it-s-pretty-easy-isn-t-it">
<h2>It's pretty easy, isn't it?</h2>
<p>Just get the newest kernel, build it for your target platform and flash it to the device. Unfortunately, it doesn't work like this in the case of phones. But if your laptop or PC is well supported by the Linux kernel, these are the only steps necessary to get the latest kernel working. Some GNU/Linux distributions even provide backports of recent kernel versions, but they normally add a few patches on top of the mainline kernel.</p>
<p>The situation is completely different with practically all Android-based smartphones. Google maintains a <a class="reference external" href="https://android.googlesource.com/kernel/common/">common Android kernel tree</a> that contains lots of patches to make the kernel work with the Android user space. There is an ongoing effort to get these patches into the mainline kernel and the situation actually looks pretty good nowadays. You only need a few patches on top of the mainline kernel to be compatible with the most relevant parts of the Android user space. The common Android kernel is based on an older kernel version that has long-term support. So the only challenge is to get a few Android-specific patches working with the latest kernel or we could stay with the common Android kernel and only have to build it, right?</p>
<p>Again, no, it doesn't work like this. If you are lucky, you will see some error messages on the screen and maybe a few other signs of life. The reason is that the phone manufacturers add a huge amount of out-of-tree code to the kernel to get support for their hardware. The added code mainly consists of drivers and support for the SoC. And as the manufacturers also base their changes on an old long-term support kernel version at the beginning of the development, the kernel that comes with the phone on the release date is already pretty out-of-date. <a class="reference external" href="https://lwn.net/Articles/662147/">1-3 million lines of out-of-tree code are normal for a smartphone</a>. So this is the point where it becomes really difficult to get a newer kernel working. And it is the reason why the manufacturers themselves don't bother updating their kernels to a newer version after the release.</p>
</div>
<div class="section" id="kernel-version-3-0-make-that-old">
<h2>Kernel version 3.0 - make that old</h2>
<p>The latest released kernel source code from Samsung for the Galaxy S3, that I am aware of, contains a kernel based on the 3.0.31 kernel version. This version was released on <a class="reference external" href="https://lwn.net/Articles/496153/">7 May 2012</a>. The phone became first available on <a class="reference external" href="https://en.wikipedia.org/wiki/Galaxy_S3">29 May 2012</a>. So besides some patches here and there, Samsung has probably never updated the kernel, not even to a newer minor release. CyanogenMod merged the latest release that is available for the 3.0.x kernel series. This updated the kernel to the 3.0.101 version which was released on <a class="reference external" href="https://lwn.net/Articles/571242/">22 Oct 2013</a>. So the kernel is only a little more than three years behind. And by the way, I even found a few remains from the 2.6 kernel when poking through the source code.</p>
<p>The difference in age is even worse when you want to port the out-of-tree code to a newer kernel version or if you want to merge a more recent kernel version because then the release date of the 3.0 kernel becomes relevant which is <a class="reference external" href="https://lwn.net/Articles/452531/">21 Jul 2011</a>, more than five years ago. The minor releases in the 3.0.x branch only include backports of important fixes from later versions. So the kernel doesn't include any of the substantial changes that were made to the Linux kernel in the last five years, except for a few changes that were backported by Samsung or CyanogenMod. The backports make it actually more difficult or outright impossible to merge a newer kernel, e.g. a 3.2.x kernel version, because some fixes were adapted for the older kernel and cause conflicts when merging a newer kernel.</p>
<div class="section" id="merging-a-newer-kernel">
<h3>Merging a newer kernel</h3>
<p>There is another reason that makes merging with newer kernel releases very difficult. A version control system like Git expects a common ancestor when merging two branches. But Samsung only releases archives of the source code without the Git history. Samsung is not required by the GPLv2 licence to release the Git history but it would be very helpful. Google publishes Git repositories of the kernels for their Nexus devices, but I am not aware of any project supporting Nexus devices that managed to merge major kernel releases.</p>
<p>Without the Git history, Git has a very hard time to resolve conflicts when merging and you end up with a huge amount of conflicts so that it's just too much work to go through all of them. This happened when I merged with the 3.2.x mainline kernel tree. <a class="reference external" href="https://code.fossencdi.org/kernel_samsung_smdk4412.git/log/?h=migrate_3.2">I was able to get a partial merge working</a>. The end result is not maintainable because there is too much unmerged code and you end up in the middle between two kernel versions. Merging more recent kernel versions becomes even more harder and you will end up with numerous bugs that no one else has. <a class="reference external" href="https://github.com/dorimanx/Dorimanx-SG2-I9100-Kernel">Dorimanx targeted the Galaxy S2 and he did partial merges up to Linux 3.14</a>, but he stopped working on it in 2014. The Galaxy S2 (GT-I9100) uses the same kernel sources.</p>
<p>In the end, all of these efforts don't really bring a newer kernel to the device and we end up with an unmaintainable mess. The goal should be to have an easily maintainable kernel that makes future kernel updates possible. In the next post, I will explain why going mainline is the only solution and how far I've got with getting the mainline kernel working.</p>
</div>
</div>
U-Boot for the Galaxy S3: an investigation2017-02-02T17:31:00+01:002017-02-02T17:31:00+01:00Wolfgang Wiedmeyertag:blog.fossencdi.org,2017-02-02:/u-boot-galaxys3.html<p class="first last">The Galaxy S3 comes with a bootloader named S-Boot which is a proprietary bootloader from Samsung. In this post, I want to collect information about the possibilty to run the free bootloader U-Boot on the phone.</p>
<p>The Galaxy S3 (GT-I9300) comes with a bootloader named S-Boot which is a proprietary bootloader from Samsung. In this post, I want to collect information about the possibilty to run the free bootloader U-Boot on the phone. I personally didn't work on the bootloader part, but I became interested in it when I tried to get the mainline kernel working and S-Boot prevented the mainline kernel from booting due to a bug.</p>
<div class="section" id="benefits">
<h2>Benefits</h2>
<p><a class="reference external" href="http://www.denx.de/wiki/U-Boot/">U-Boot is free software</a>. If you want to run as much free software as possible on your phone, then being able to replace the proprietary bootloader with a free one would obviously be great. A free bootloader would make it possible to control and verify which software is loaded during the boot process. In case of the Galaxy S3, it turns out that not only the Linux kernel and subsequently Android is loaded, but also a second proprietary operating system, a <a class="reference external" href="https://redmine.replicant.us/issues/1659#note-5">TrustZone implementation called Mobicore</a> and it's not fully clear what Mobicore does. A free bootloader could make it possible to decide if something like Mobicore should be loaded or not.</p>
<p>A free bootloader would also make new security features like verified boot possible. The bootloader could check the signature of the kernel and the kernel could verify the integrity of the whole system. This way, it would be possible to ensure that no one and nothing has tampered with the operating system. More control over the boot process could also lead to new features that would make it a lot easier to unbrick devices that refuse to boot. For example an option that allows to boot from the SD card would be very helpful in this regard.</p>
</div>
<div class="section" id="is-it-possible">
<h2>Is it possible?</h2>
<p>In 2013, Dominik Marszk and Adam Outler managed to boot U-Boot from the SD card. Their U-Boot source can be found <a class="reference external" href="https://github.com/Rebell/exynos4_uboot">here</a>. The Galaxy S3 tries to boot from eMMC (internal memory) first and if that fails, it attempts to boot from other possible boot devices like the SD card. In order to force booting from SD card, Adam corrupted the data transfer between CPU and eMMC by attaching a thin wire to one of the data lines that shortens the data line. After U-Boot has booted, it enters a 10 second delay to allow to release the line shortening. At this point, it's possible to talk to the U-Boot command line via UART.</p>
<p><a class="reference external" href="https://github.com/Rebell/exynos4_uboot/tree/master/sd_fuse">These</a> are the files they used to create the SD card. They come from <a class="reference external" href="http://www.hardkernel.com/main/products/prdt_info.php?g_code=G133999328931">ODROID-X</a> bootloader sources which are based on Exynos4412, just like the Galaxy S3. The binary p4412_s_fwbl1.bin is especially interesting because it is a first stage bootloader that drops the secure boot process and jumps to an unsigned payload. This signed first stage bootloader also works on the Galaxy S3 because the same signing key is fused into all Exynos4412-based development boards and handhelds.</p>
<p>Dominik's and Adam's main problem was that they couldn't get a stock kernel image to boot. The cause was likely related to the TrustZone not getting initialized properly. They never tried a modified kernel, so this could be a place to start. <a class="reference external" href="https://code.fossencdi.org/kernel_samsung_smdk4412.git/commit/?h=replicant-6.0&id=7fbe662a46f3bb994b6f7a9adea731f3d8a5620c">I disabled the Mobicore driver</a> in the kernel for <a class="reference external" href="https://blog.replicant.us/2016/08/replicant-6-early-work-upstream-work-and-f-droid-issue/">Replicant 6.0</a> in an attempt to get rid of any cooperation between the kernel and TrustZone. It would be interesting if it works with this kernel.</p>
<p>The advantage of Dominik's and Adam's approach is the possibility to replace the proprietary bootloader S-Boot with U-Boot. A proprietary first stage bootloader is still necessary. And the second stage bootloader is created by a <a class="reference external" href="https://github.com/Rebell/exynos4_uboot/blob/master/mkbl2">proprietary tool mkbl2</a> that is part of the U-Boot sources and was originally supplied by Samsung. The main disadvantage of their approach is the necessity to corrupt the data transfer between the CPU and eMMC which requires experience, is only suitable for development and testing purposes and has a high risk to render the device unusable if done wrong.</p>
<p>There is some info floating around about a recovery procedure that boots from the SD card, but replaces the existing bootloader which resides in the bootloader partition on eMMC. <a class="reference external" href="http://forum.xda-developers.com/showpost.php?p=47234165&postcount=220">There are claims</a> that it's possible to trigger this process with more recent Galaxy S3 devices by pressing the menu key, both volume keys and the power key. Otherwise, it seems to be necessary to disassemble the device and shorten a very small resistor with a pair of tweezers, according to <a class="reference external" href="https://smyl.es/samsung-galaxy-iii-s3-gt-i9300-jtag-leaked-document-how-to-repair-soft-bricked-galaxy-s3/">leaked Samsung documents</a>. This procedure could be an alternative to corrupting the data transfer between CPU and eMMC.</p>
<p>As part of my work with the mainline kernel on the Galaxy S3, I submitted a <a class="reference external" href="https://patchwork.kernel.org/patch/9345815/">bootloader-related patch that allows to reboot the device in the recovery and download mode</a>. In the discussion about the patch with the kernel maintainer Krzysztof Kozlowski, it turned out that Trats2, which is the Galaxy S3 released for Tizen, is supported by U-Boot. However, S-Boot was not replaced. Instead, U-Boot gets chainloaded after booting a (probably) adapted S-Boot image. <a class="reference external" href="http://www.spinics.net/lists/arm-kernel/msg534042.html">Krzysztof's mail</a> provides more info how the Trats2 image could be installed.</p>
<p>Freedom-wise, chainloading U-Boot after S-Boot is not attractive as no non-free software is replaced. But having the Linux kernel loaded by U-Boot would make working on the mainline kernel easier because I don't have to figure out all the differences between U-Boot and S-Boot and patch the kernel. The only question in this regard is whether it's less work to stick with S-Boot and work around the differences or to get U-Boot for Trats2 working with a Galaxy S3 targeting Android and avoid having to patch the kernel.</p>
<p>In the end, it would obviously be most rewarding if it's possible to replace S-Boot. If a signature check is enforced on S-Boot and the first and second stage bootloader are not replaceable, then there is no hope, unless the signing key gets leaked. If only the first and second stage bootloader need to be signed and S-Boot runs outside of the secure boot process, then maybe it's possible to get U-Boot working while the proprietary first and second stage bootloader stay in place. Maybe there is also some crazy way to make booting from SD card permanent while still having the internal memory accessible. So there are still some basic questions that need to be investigated.</p>
<p>I want to thank Dominik Marszk, Adam Outler and Krzysztof Kozlowski for providing lots of useful information which made this post possible.</p>
</div>
Nonfree firmwares for Android devices2017-01-07T22:27:00+01:002017-03-04T19:30:00+01:00Wolfgang Wiedmeyertag:blog.fossencdi.org,2017-01-07:/nonfree-firmware-android.html<p class="first last">The following instructions make it possible to install nonfree firmware on Android devices.</p>
<p>The following instructions make it possible to install nonfree firmware on Android devices for the purpose of enabling certain functionalities. Please see <a class="reference external" href="http://code.paulk.fr/article16/missing-proprietary-firmwares-in-android-systems">Paul's blog post</a> for the reasons why these instructions shouldn't be published on official documentation pages of free systems, but why it still makes sense to publish them elsewhere.</p>
<p>Paul's script is compatible with CyanogenMod versions 10.1.3 and 9.1.0. It extracts the firmwares from a CyanogenMod installation zip and either installs them with ADB or creates a new installation zip with them. My implementation is intended to be compatible with LineageOS/CyanogenMod 13.0. It's possible to only select a certain functionality for which the firmwares should be installed. The firmwares are downloaded from a <a class="reference external" href="https://github.com/TheMuppets">repository</a>, that hosts the firmware files for CyanogenMod builds, and they are installed with ADB.</p>
<p>Downloading the firmware files seemed to be the most straightforward solution since it's impractical to extract them from a CyanogenMod 13.0 installation zip because recent Android versions use complicated compression techniques that require tools which are usually not readily available as packages for GNU/Linux systems. Creating installation zips is also not easy anymore because the zip needs to be signed with a key that is trusted by the recovery. If your recovery still accepts any installation zip, then you are using an insecure recovery.</p>
<p>Make sure that you have ADB and root access with ADB enabled.
These steps are required:</p>
<ol class="arabic">
<li><p class="first">Clone the <a class="reference external" href="https://code.fossencdi.org/firmwares_nonfree.git/">firmwares_nonfree repository</a>:</p>
<div class="highlight"><pre><span></span><span class="gp">$</span> git clone https://code.fossencdi.org/firmwares_nonfree.git
</pre></div>
</li>
<li><p class="first">Connect the device to your PC and check if the device is detected:</p>
<div class="highlight"><pre><span></span><span class="gp">$</span> adb devices
</pre></div>
<p>If the list of attached devices is empty, you will have to check your ADB setup first and figure out why the device doesn't get recognized.</p>
</li>
<li><p class="first">Run the script in the git repository with two arguments, the name of your device and the functionality you want to enable, e.g.:</p>
<div class="highlight"><pre><span></span><span class="gp">$</span> ./firmwares.sh i9300 wifi
</pre></div>
<p>This command will only install the firmware files that are needed to enable wifi. The firmware will be downloaded, pushed to the device and the device is rebooted. Use <tt class="docutils literal">list</tt> as second argument to get a list of functionalities that can be enabled for the specific device. You can also specify <tt class="docutils literal">all</tt> to install all firmware files which will enable all listed functionalities.</p>
</li>
<li><p class="first">If you want to remove a certain firmware again, you can do so by adding <tt class="docutils literal">remove</tt> as third argument, e.g.:</p>
<div class="highlight"><pre><span></span><span class="gp">$</span> ./firmwares.sh i9300 front-camera remove
</pre></div>
<p>This command deletes the firmware files for the front camera from the device and reboots it.</p>
</li>
</ol>
Setting up a Debian Stretch build environment for Replicant 6.0 using debootstrap2016-12-28T23:10:00+01:002016-05-07T18:10:00+02:00Wolfgang Wiedmeyertag:blog.fossencdi.org,2016-12-28:/debian-stretch-debootstrap-replicant.html<p class="first last">Recently, I managed to to make <a class="reference external" href="https://blog.replicant.us/2016/08/replicant-6-early-work-upstream-work-and-f-droid-issue/">Replicant 6.0</a> buildable on Debian Stretch. I'm still using Debian Jessie on my machines, so I set up a chroot with Stretch. I'll document all the necessary steps in the following to establish a Stretch build environment and build Replicant 6.0 inside it.</p>
<p>Recently, I managed to make <a class="reference external" href="https://blog.replicant.us/2016/08/replicant-6-early-work-upstream-work-and-f-droid-issue/">Replicant 6.0</a> buildable on Debian Stretch. This makes it possible to use a lot of new Android-related packages in Debian. I'm still using Debian Jessie on my machines, so I set up a chroot with Stretch. I'll document all the necessary steps in the following to establish a Stretch build environment and build Replicant 6.0 inside it. This should work across many distros, although I only tested it on Debian Jessie. The only tool you'll need is a <a class="reference external" href="https://wiki.debian.org/Debootstrap">debootstrap</a> that contains the necessary scripts to set up a Debian Stretch chroot.</p>
<div class="section" id="installing-debian-stretch-in-a-chroot-with-debootstrap">
<h2>Installing Debian Stretch in a chroot with debootstrap</h2>
<p>First install debootstrap. I recommend getting the latest version that is available for you, e.g. from <a class="reference external" href="https://packages.debian.org/jessie-backports/debootstrap">Jessie Backports</a>. Then create a directory where you want the chroot to be.</p>
<div class="line-block">
<div class="line"><cite>If you are using a kernel with grsecurity</cite>:</div>
<div class="line"><a class="reference external" href="https://grsecurity.net/">Grsecurity</a> does quite some hardening of chroots. Unfortunately, we need to disable a few of the configuration options to make debootstrap work. You will need to deactivate the following three in /etc/sysctl.d/grsec.conf:</div>
<div class="line"><tt class="docutils literal">kernel.grsecurity.chroot_deny_chmod</tt></div>
<div class="line"><tt class="docutils literal">kernel.grsecurity.chroot_caps</tt></div>
<div class="line"><tt class="docutils literal">kernel.grsecurity.chroot_deny_mount</tt></div>
<div class="line">Depending on your configuration, a reboot might be necessary.</div>
<div class="line"><br /></div>
</div>
<p>Then you can run debootstrap to set up a minimal Stretch install. The command needs to be run as root, while <tt class="docutils literal">$YOUR_CHROOT_DIR</tt> is the absolute path to the directory you created for the chroot:</p>
<div class="highlight"><pre><span></span><span class="gp">#</span> debootstrap stretch <span class="nv">$YOUR_CHROOT_DIR</span>
</pre></div>
<p>The proc filesystem needs to be mounted inside the chroot:</p>
<div class="highlight"><pre><span></span><span class="gp">#</span> mount proc <span class="nv">$YOUR_CHROOT_DIR</span>/proc/ -t proc
</pre></div>
<p>Every time you enter the chroot to build Replicant 6.0, you will need to mount the proc filesystem. So I recommend adding a permanent entry in your <tt class="docutils literal">/etc/fstab</tt>:</p>
<pre class="literal-block">
proc $YOUR_CHROOT_DIR/proc proc defaults 0 0
</pre>
<p>If you don't mount the proc filesystem, you will get very weird build errors and it might take some time until you figure out that it's because proc is not mounted. Trust me, I was already there.</p>
<p>For initial setup, devpts also needs to be mounted:</p>
<div class="highlight"><pre><span></span><span class="gp">#</span> mount --bind /dev/pts <span class="nv">$YOUR_CHROOT_DIR</span>/dev/pts
</pre></div>
<p>Then you can finally enter the chroot:</p>
<div class="highlight"><pre><span></span><span class="gp">#</span> chroot <span class="nv">$YOUR_CHROOT_DIR</span>
</pre></div>
<p>Tell apt where it should get the source packages from:</p>
<div class="highlight"><pre><span></span><span class="gp">#</span> <span class="nb">echo</span> <span class="s2">"deb-src http://deb.debian.org/debian stretch main"</span> >> /etc/apt/sources.list
</pre></div>
<p>Replacing <tt class="docutils literal">deb.debian.org</tt> in your <tt class="docutils literal">sources.list</tt> with a local mirror likely leads to a lot faster downloads and puts less load on the Debian servers. I'm in Germany so I have <tt class="docutils literal">ftp.de.debian.org</tt> in my <tt class="docutils literal">sources.list</tt>.</p>
<p>Then add the i386 architecture and update:</p>
<div class="highlight"><pre><span></span><span class="gp">#</span> dpkg --add-architecture i386
<span class="gp">#</span> apt-get update
</pre></div>
<p>Before installing all the necessary build dependencies, locales needs to be configured correctly:</p>
<div class="highlight"><pre><span></span><span class="gp">#</span> apt-get install locales
<span class="gp">#</span> dpkg-reconfigure locales
</pre></div>
<p>Then install <a class="reference external" href="https://redmine.replicant.us/projects/replicant/wiki/BuildDependenciesInstallation#Debian-based-systems">all dependencies</a>.</p>
<p>Add a user for building Replicant 6.0:</p>
<div class="highlight"><pre><span></span><span class="gp">#</span> adduser <span class="nv">$YOUR_USER</span>
</pre></div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><cite>If you are using a kernel with grsecurity</cite>:</div>
<div class="line">In case Trusted Path Execution (TPE) is enabled, you will have to recreate the group inside the chroot and add your user to it:</div>
<div class="line">groupadd -r -g 64040 grsec-tpe</div>
<div class="line">usermod -aG grsec-tpe $YOUR_USER</div>
<div class="line"><br /></div>
</div>
<p>Switch to your user:</p>
<div class="highlight"><pre><span></span><span class="gp">#</span> su <span class="nv">$YOUR_USER</span>
</pre></div>
<p>From now on, you can follow the <a class="reference external" href="https://redmine.replicant.us/projects/replicant/wiki#Replicant-build">respective build page for your device</a> (given that Replicant 6.0 is supported on your device).</p>
<p>Some <tt class="docutils literal">repo</tt> commands fail if the shared memory is not mounted inside the chroot. So if you use <tt class="docutils literal">repo</tt> from inside the chroot, mount <tt class="docutils literal">dev/shm</tt>:</p>
<div class="highlight"><pre><span></span><span class="gp">#</span> mount --bind /dev/shm <span class="nv">$YOUR_CHROOT_DIR</span>/dev/shm
</pre></div>
</div>
Git Hosting: Reducing Server Load with Bundles2016-12-18T23:40:00+01:002016-12-18T23:40:00+01:00Wolfgang Wiedmeyertag:blog.fossencdi.org,2016-12-18:/git-bundle-hosting.html<p class="first last">Cloning large repositories can result in quite some load on the server side. A possible solution is the use of bundles. Git can package a certain revision in an archive. The client can fetch the bundle and set up a clone locally based on the bundle.</p>
<p>Cloning large repositories can result in quite some load on the server side. Depending on the server specs, the server may run out of RAM or the CPU load increases heavily. In my case, the limiting factor is the CPU. Too much load can even result in fatal errors that make it impossible to freshly clone a repository.</p>
<p>A possible solution is the use of bundles. Git can package a certain revision in an archive. The client can fetch the bundle and set up a clone locally based on the bundle. The <a class="reference external" href="https://git-scm.com/docs/git-bundle">Git documentation</a> describes how this works. The server then has the only task of providing the bundle which requires almost no load. When the client has set up the clone with the bundle, subsequent pull or fetch requests will take a lot less server load because the server only needs to handle the diff between the revision archived in the bundle and the revision that currently is fetched.</p>
<p>The Linux kernel project uses bundles on their Git hosting servers and <a class="reference external" href="https://www.kernel.org/cloning-linux-from-a-bundle.html">they recommend to directly get the bundle with wget if you have connection problems</a>. The <a class="reference external" href="https://code.google.com/p/git-repo/">repo tool</a>, which manages the various Git repositories of Android-based operating systems, by default even expects that a bundle with the name clone.bundle is present in every repository on the server during the initial sync. The repo tool automatically fetches the bundles and uses them to set up the individual Git repositories.</p>
<div class="section" id="creating-bundles-on-the-server">
<h2>Creating bundles on the server</h2>
<p>Bundles are easily created inside a Git repository with the command <tt class="docutils literal">git bundle create clone.bundle $REVISON</tt>. <tt class="docutils literal">$REVISION</tt> can be a branch or a tag. In case you have a lot of Git repositories and if all of them are in the same directory, running the following command in the parent directory may be helpful to create bundles in all of them:</p>
<div class="highlight"><pre><span></span><span class="k">for</span> i in *.git<span class="p">;</span> <span class="k">do</span> <span class="o">(</span> <span class="nb">echo</span> <span class="nv">$i</span><span class="p">;</span> <span class="nb">cd</span> <span class="nv">$i</span><span class="p">;</span> git bundle create clone.bundle <span class="nv">$REVISION</span><span class="p">;</span> <span class="o">)</span><span class="p">;</span> <span class="k">done</span>
</pre></div>
</div>
<div class="section" id="making-the-bundles-accessible">
<h2>Making the bundles accessible</h2>
<p>If you do your own Git hosting, you probably have a web server like Apache running and some software like <a class="reference external" href="https://git.zx2c4.com/cgit/">cgit</a> serves as Git web frontend behind the web server. As I'm using <a class="reference external" href="http://gitolite.com/gitolite/">Gitolite</a> to manage access to my repositories, all repositories reside in the directory <tt class="docutils literal">/var/lib/gitolite3/repositories</tt>.</p>
<p>First, Apache needs to be told where it can find the bundles:</p>
<div class="highlight"><pre><span></span><span class="nb">AliasMatch</span> ^/(.*).git/clone.bundle <span class="sx">/var/lib/gitolite3/repositories/</span>$1.git/clone.bundle
<span class="nb">AliasMatch</span> ^/(.*)/clone.bundle <span class="sx">/var/lib/gitolite3/repositories/</span>$1.git/clone.bundle
</pre></div>
<p>These directives make sure that regardless of whether the URL contains the <tt class="docutils literal">.git</tt> suffix, Apache finds the corresponding <tt class="docutils literal">*.git</tt> folder.</p>
<p>Then clients need to be allowed to access the bundles in the git repositories:</p>
<div class="highlight"><pre><span></span><span class="nt"><Directory</span> <span class="s">/var/lib/gitolite3/repositories/</span><span class="nt">></span>
<span class="nb">Require</span> <span class="k">all</span> denied
<span class="nt"><FilesMatch</span> <span class="s">"clone.bundle"</span><span class="nt">></span>
<span class="nb">Require</span> <span class="k">all</span> granted
<span class="nt"></FilesMatch></span>
<span class="nt"></Directory></span>
</pre></div>
<p>This ensures that only files named clone.bundle are accessible.</p>
<p>I hope that having bundles available causes a lot less issues when syncing with my <a class="reference external" href="https://blog.replicant.us/2016/08/replicant-6-early-work-upstream-work-and-f-droid-issue/">Replicant 6.0</a> repositories or with my <a class="reference external" href="https://replicantmirror.fossencdi.org/">Replicant 4.2 mirror</a>.</p>
</div>
Why is our society so segregated?2016-08-16T23:12:00+02:002016-09-11T22:16:00+02:00Wolfgang Wiedmeyertag:blog.fossencdi.org,2016-08-16:/why_segregated_society.html<p class="first last">It's 2016 and not much seems to have changed over the last decades when looking at the big picture. Very few people are getting richer and richer while the rest has less and less. There are many violent conflicts around the world. People are divided by religion, race, ...</p>
<p>It's 2016 and not much seems to have changed over the last decades when looking at the big picture. Very few people are getting richer and richer while the rest has less and less. There are many violent conflicts around the world. People are divided by religion, race, line of descent and social standing. People don't even understand the concerns of others and often political discussions cannot even be started because one side already feels offended when the other side disagrees with them. I thought that further improvements to our political systems, a free press and the Internet would change a lot for the better, so we as a society could work together on deciding what issues matter most and get rid of them one after the other. But there are many negative developments that slow down the process and they could even invert a lot of positive change.</p>
<p>I blame a lot on the bad political elite we currently have almost everywhere. Most people are not even interested in politics anymore and they don't see themselves represented. They rather spend they spare time with various distractions. I consider the biggest achievement of chancellor Merkel that now a whole young generation in Germany is completely disenchanted with politics. If people are going to vote, they are not passionately casting their vote for their favorite candidate but for the candidate that will probably cause less damage than the others. The politicians in power won't change anything about this because this is exactly how they want it. A disinterested population is the best population for a ruling party in a democracy and they can do what they want: either nothing or that what the currently highest bidder wants. However, they should bring the population together and cause progress.</p>
<p>Take a look at the United States presidential elections: Among the American population of over 300 million people, the electoral system of the two biggest parties managed to designate two of the worst possible candidates: a gabbling populist and the personification of corruption. One candidate represents a dangerous group of people that is fueled by hate, distrust in the political elite and that believes that all their problems can be solved by oversimplified solutions at the cost of others. The other candidate represents the status quo, a small group of people in Washington that is busy meeting lobbyists all day, that won't be prosecuted of any crime they commit and that discusses the next violent regime change in their cozy think tank conference rooms. But the real scandal about the presidential elections are not these two ambassadors of the apocalypse, but that most Americans don't know that there are more than two candidates running for president. This is due to the media that is not reporting on them.</p>
<p>And this brings us to the next issue. You may think that a free press gives our politicians a hard time and informs us in a neutral way about relevant issues. This may be true in many cases and there are many talented and passionate journalists out there working hard for all of us. But the press cannot be completely neutral or objective. The selection of issues to report about already intentionally or unintentionally pushes a certain agenda. If a newspaper doesn't report on an event, then this event never happened to the readers of the newspaper. Then there are various conflicts of interests if many newspapers and publishing houses are owned by rich and influential individuals or families. Furthermore, bringing the news first seems to be most important nowadays, so the quality of the reporting is often low and newspapers are more or less copying the same article from one of the few news agencies. All of this makes it hard to get different perspectives about the same issue and to form an informed opinion.</p>
<p>There is another amplifying development: The mainstream media is more and more filling the void that religion has left in our secular world. It is important to stick to a certain message. The message should be short and broadcast by everybody in the same way, so we all have something to believe in and don't have to worry about the real complexity of the problems we are facing. Facts are not that important anymore and are ignored if they don't fit the narrative. Whoever is casting doubt at the current narrative, is called a conspiracy theorist, even if she or he is not even proposing a theory. This is in no way different than the propaganda during the Cold War. The "shooting the messenger" principle is also used frequently. Let's attack the bearer of the information if we don't like what he is telling us. Among others, Wikileaks published a huge amount of emails that shows corruption among the leaders of the Democratic Party and collusion with the Clinton campaign. The media rather speculates about the possibility that the data was originally obtained by people that work for Putin. Edward Snowden showed us widespread surveillance and illegal activity of the NSA and others. Let's rather talk about the theory that Snowden is an agent working for Putin. By the way, this childish Putin blaming also leads to less detailed reporting about issues inside and outside Russia that are caused by president Putin because he is already established as the personified evil.</p>
<p>This behavior of the media boosts the societal segregation. Everyone that keeps in line will have difficulties to form their own opinion due to lack of details and perspectives. Furthermore, she or he is encouraged to not worry and do something about our problems because there are already good people working on solving them. Everyone who comes to different conclusions and wants a discussion is alienated by hostility. This leads to the many forms of radicalization we see today. People are breaking out of the bubble. But instead of trying to get a more complete world view, they seek a new bubble where everyone has their opinion and detests everyone else that doesn't have their opinion. They are just following the path of least resistance.</p>
<p>I hoped that the Internet would change something about this. In fact, the group of the well informed seems greater than ever and investigative journalists have numerous sources. For activists it is much easier to organize themselves. But it's also easier than ever to find someone that confirms the craziest claims and biggest lies. Sources are either not checked at all or in a very sloppy way. If someone we know and trust shares a story on social media, then it has to be true and we don't need to read more than the headline. So while the internet fosters knowledge and communication, it also helps to gather people in their own little radical groups and pushes us more apart. Maybe the Internet is just too complex or it still costs too much effort to do proper research on current topics?</p>
<p>Maybe people need to be taught how to do the research itself and handle the vast amounts of different sources we have today. For sure, lack of time plays a role, too. Besides all the working and self-optimizing we have to do, there are not much time and mental resources left to follow up what is going on. More political influence for everyone in the sense of direct democracy may also improve the situation, but only if everyone is well educated on the issues and seeks discourse. Otherwise, it will probably lead to more uninformed decisions, more frequent policy changes and more harm to everyone.</p>
<p>I'm not sure what exactly needs to be done. There are obviously more causes than politics and media. The situation is complex and I cannot come up with a simple solution that can be broadcast as a short message.</p>
Why I use free software and why it's important to me2016-08-14T20:20:00+02:002016-09-11T22:50:00+02:00Wolfgang Wiedmeyertag:blog.fossencdi.org,2016-08-14:/why_free_software.html<p class="first last">I try to use as much <a class="reference external" href="https://www.gnu.org/philosophy/free-sw.html">free software</a> as possible. I don't only do this for moral or philosophical reasons, but also because there are numerous practical benefits.</p>
<p>I try to use as much <a class="reference external" href="https://www.gnu.org/philosophy/free-sw.html">free software</a> as possible. You may have heard the term "open source" for what I call free software. <a class="reference external" href="https://www.gnu.org/philosophy/open-source-misses-the-point.html">Others</a> and I don't use this term as it puts the emphasis only on one property of the software and not on the part that is most important to me: freedom. Free software means that you can use the software for any purpose, modify it and share the software, either in unmodified or modified form. I don't only use free software for moral or philosophical reasons, but also because there are numerous practical benefits. I may have some disadvantages or inconveniences by trying to not use proprietary or nonfree software. It may even prevent me from using certain hardware if it's not possible at all to use this hardware with free software. But many of the problems turn out to be issues only at first sight or are outweighed by the advantages of their free counterparts. You can check for yourself if the software you are using is free software. Check the license it was published under. It should be one of the <a class="reference external" href="https://www.gnu.org/licenses/license-list.en.html#SoftwareLicenses">free licenses</a>.</p>
<div class="section" id="giants">
<h2>Giants</h2>
<p>Free software is the future. There are thousands of free software developers around the world and you are really standing on the shoulders of giants when you are using free software. Companies or individuals that produce proprietary software cannot compete in the long run with these development efforts of a sharing and cooperating community. I use <a class="reference external" href="https://www.debian.org/">Debian</a> as operating system on my PC, laptop and servers. Debian has <a class="reference external" href="https://www.debian.org/intro/about">around a thousand developers</a>. Nearly all of these developers don't actually develop the software they maintain, but they fix issues here and there and make sure that everything works together nicely. Debian does not only offer you the base system like Windows or Mac OS does, but it already includes nearly all the software you need, more than <a class="reference external" href="https://www.debian.org/intro/about">43.000 individual software packages in total</a>. A common myth is that free software is only based on the efforts of hobbyists, but in fact a lot of free software developers are professionals that make their living by creating free software and there are quite some companies that make money from developing free software or providing support for it. For example, more than <a class="reference external" href="https://www.linuxfoundation.org/news-media/announcements/2015/02/linux-foundation-releases-linux-development-report">eighty percent of the several thousand Linux kernel developers get paid for their work on the kernel</a>. The Linux kernel is the core of a GNU/Linux operating system like the one that is offered by Debian.</p>
</div>
<div class="section" id="security-and-privacy">
<h2>Security and privacy</h2>
<p>Real security and privacy is only possible with free software. Free software is the prerequisite to have security and privacy in our computing. Only free software can be fully vetted by anybody. Security researchers can independently audit the software and share their results. Everybody can work on improving the software and make it more trustworthy. And the users can ensure themselves that the software they are running is actually doing what it's supposed to be doing. Developers of nonfree software often solely rely on security by obscurity. They think that if they hide the source code or, in general, information on the inner workings of the software, then nobody can find vulnerabilities or backdoors in the software. This behavior is comparable to children who cover their eyes and think they are invisible. People still have a lot of ways to identify security issues in the software and steal your data or listen in on your laptop's microphone. Features that harm the users won't survive in free software. Someone will get rid of them and the users will start to use the version of the software that has these antifeatures removed.</p>
<p>You don't have to be a software developer or a security expert. Alongside the free software developer community comes a huge community of very nice people that will help you with numerous tutorials and documentation. You will most certainly also find someone who answers your questions directly. Even if you have no interest in any of the technical details, you will find plenty of support to use your computer securely and retain your privacy.</p>
<p>My understanding of privacy is that I can decide what information I share with whom. My goal is not to run around as a huge question mark, my goal is to be in control of my own data and in the broader sense of my own destiny. Only free software has the possibility to guarantee this control in the digital age of computers. Even if you don't value your own security or privacy for whatever reason, you should acknowledge the need of others. If someone asks you for a private conversation, you will probably step aside, lower your voice and listen to what the other one has to say. The same goes for mail exchange, the chat software you use with your friends or the online service you use to exchange files with others. If some of your peers care for their privacy or security, then you should respect that. It won't hurt you to do something to increase the level of your privacy and security, at least for the sake of others.</p>
<p>There is a technological area where I especially want to see the use of free software: medical devices. If my well-being depends on the well-functioning of a device like a pacemaker or an operational tool for a surgeon, then I want to have a look at the source code that runs on that device. I want to show the source code to experts and they should be able to tell me if the software works correctly. If there is a way to update the software on the device and if the software running on the device needs to be improved, then I should have the right to let someone do the update. Companies that produce medical devices should be held to the highest possible standards of quality assurance. Publishing the software for medical devices as free software should be a prerequisite of the quality assurance. We can't solely trust the companies because the interests of the customers is not always aligned with the business interests of the companies. If source code is not published under a free software license because certain persons in charge think that this may decrease their revenue, then the well-being of the patient is neglected merely because of the possibility that someone gets more money.</p>
<p>Our devices are getting more and more powerful. And the need for security actually grows at an even faster rate. A few years back, we were wondering if someone could access our files and manipulate the software on our PC if we connect our PC to the Internet. Then we feared that we were watched through our webcam. After that, smartphones became a thing and we started to carry around wiretaps that can also track our locations. As security is more and more lacking on these devices, the gap between increased abilities of these devices and the need to secure the devices is becoming bigger and bigger. Especially on mobile devices, the lack of security updates or of any effort to make them trustworthy is sometimes outright horrifying. This is one reason why I try to contribute to the <a class="reference external" href="http://www.replicant.us/">Replicant project</a>.</p>
<p>But smartphones are not the end. We are more and more entering the reality of the buzzword of nightmares: the internet of things. Why not connect everything to the Internet? At some point in the future, almost every car will have some sort of remote control. There's a real possibility that you will buy for your parents a robot that will help them in the household when they get older. So the thread of being passively watched, listened to or tracked will be extended by the thread of being actively physically harmed. There are already numerous reports about the lack of software security in cars and security researchers were able to remotely control cars. I don't want to live in a world where some madman can sit on his bed, sip his coffee and occasionally crash my car or choke my parents to death with the robot I bought for them. With free software, we are be able to see what functionality the devices have, how its security can be improved or more simple: We would be able to disable a certain functionality like remote control if it's just ridiculous to have that functionality in the first place.</p>
<p>Our world and our society is not getting improved by people who say that everything is alright. People who call out the issues at hand and nag us about them are doing the first step to change something for the better. Actually doing something to solve our problems is the next step. Activists are doing all of this. Whistleblowers provide us with the information. Investigative journalists research the issues. All of these groups highly depend on free software for their daily tools. Free software makes it possible for them to evade surveillance of authoritarian or corrupt governments. For some of them, security or privacy issues in the software they use can cost them their lives. I see the striving for more transparency and the opposition to censorship as one of the biggest tasks for our generation. If we all use free software, then it will be really difficult for evil organizations to even single out the activists or whistleblowers among us. But doctors and lawyers should demand free software, too. How should they be able to guarantee the privacy of their clients otherwise?</p>
</div>
<div class="section" id="sustainability-education-and-research">
<h2>Sustainability, education and research</h2>
<p>Free software makes sustainability possible. You can run recent software on quite old hardware because it's technically possible and there is no company hindering you from doing that because they think that this may be bad for their business model. And in the same way, you can be sure that you will be able to access your data or your work for many years to come. Many people cannot afford to buy new hardware every two years. With free software they can use their hardware without any issues for up to fifteen or more years and still benefit from new features and security updates. Using our hardware for as long as possible is also a way to protect the environment. Normally, we cannot ensure that the manufacturers adhere to current standards to protect the environment or that they respect their workers' rights and don't exploit them. A longer utilization time and buying second-hand hardware are ways to not support abuse of workers and to conserve our environmental resources. We are living in a society that promotes consumerism. Free software helps to gain a new perspective on these issues, also by encouraging the users to contribute.</p>
<p>Free software should be mandatory in education and research. Especially if I spend a lot of time learning to use certain software or if I for some reason strongly depend on a piece of software as part of my work, then I want to be able to use that software for any purpose and for as long as I want. Additionally, I want to share that software with others and fix issues or ask others to fix problems with the software so that I can continue to use it. Only in this way, the knowledge from learning to use the software is not lost and everybody can fully benefit from the software. Researchers should use free software and they should publish the code they write as part of their research as free software. This way, it's possible to reproduce their results and to build upon them.</p>
<p>Children in school shouldn't use nonfree software that hides information from them, collects their personal data and makes them dependent on the developers or the companies behind the software. Many companies that develop nonfree software give their software away at no cost to schools and universities. Later on, they can extort ridiculous amounts of money from the graduates or the employers they work for so they can continue to use the nonfree software they just learned to use with a lot of effort. In the case that the companies go bankrupt and the software development ceases, the acquired knowledge about the software would be completely lost. With free software the source code of the software is available, so we can continue to maintain the software for ourselves or anybody else can pick up the development of the software so we can continue to use it. Free software promotes collaboration and encourages to take a look inside to see how everything works. In the sense of open access to knowledge, free software allows students and others alike to discover how a computer works and what happens if we change some parts of the source code. I personally only started to enjoy using a PC in order to get work done (especially coding) as soon as I started to deliberately use free software. Meeting others of the community at gatherings like <a class="reference external" href="https://fosdem.org">FOSDEM</a> or the <a class="reference external" href="https://events.ccc.de/congress/">Chaos Communication Congress</a> increased my enthusiasm.</p>
<p>I think that free software will gain a lot more traction as soon as more people will not only value the functionality and ease of use of the software they use, but also security, privacy, sustainability and societal effects. I hope that I could give you a better understanding of these effects. All these benefits of free software stem from the freedoms we get by using free software. If you believe in a free society, then I hopefully convinced you that free software should be a part of it. We should demand these freedoms as rights for ourselves. If we don't do this, then others will be in control: governments, companies or the machines themselves.</p>
</div>