planet@commandline:~ $ _

This Grml release provides fresh software packages from Debian trixie. As usual it also incorporates current hardware support and fixes known bugs from previous Grml releases.

More information is available in the release notes of Grml 2024.02.

Grab the latest Grml ISO(s) and spread the word!

Thanks to everyone contributing to Grml and this release, stay healthy and happy Grml-ing!

We are proud to announce the first release candidate of the upcoming version 2024.02, code-named 'Glumpad'!

This Grml release provides fresh software packages from Debian trixie. As usual it also incorporates current hardware support and fixes known bugs from the previous Grml release.

For detailed information about the changes between 2022.11 and 2024.02(-rc1) have a look at the official release announcement.

Please test the ISOs and everything you usually use and rely on, and report back, so we can complete the stable release soon. If no major problems come up, the next iteration will be the stable release, which is scheduled for end of February 2024.

We are proud to announce the first release candidate of the upcoming version 2022.11, code-named 'MalGuckes'!

This Grml release provides fresh software packages from Debian bookworm. As usual it also incorporates current hardware support and fixes known bugs from the previous Grml release.

For detailed information about the changes between 2021.07 and 2022.11(-rc1) have a look at the official release announcement.

Please test the ISOs and everything you usually use and rely on, and report back, so we can complete the stable release soon. If no major problems come up, the next iteration will be the stable release, which is scheduled for end of November 2022.

Wow, how time flies! 20 years ago grml.org was registered by Mika, and in 2004 we had our first public Grml release. 🥳

We're glad about such a wonderful community and are celebrating this with a small Easter egg in our Grml daily ISOs! 😜

And now for another 20 years? 🤓

This Grml release provides fresh software packages from Debian bookworm. As usual it also incorporates current hardware support and fixes known bugs from previous Grml releases.

More information is available in the release notes of Grml 2022.11.

Grab the latest Grml ISO(s) and spread the word!

Thanks to everyone contributing to Grml and this release, stay healthy and happy Grml-ing!

This Grml release provides fresh software packages from Debian bullseye. As usual it also incorporates current hardware support and fixes known bugs from previous Grml releases.

More information is available in the release notes of Grml 2021.07.

Grab the latest Grml ISO(s) and spread the word!

Thanks to everyone contributing to Grml and this release, stay healthy and happy Grml-ing!

We are proud to announce the first release candidate of the upcoming version 2021.07, code-named 'JauKerl'!

This Grml release provides fresh software packages from Debian bullseye. As usual it also incorporates current hardware support and fixes known bugs from the previous Grml release.

For detailed information about the changes between 2020.06 and 2021.07(-rc1) have a look at the official release announcement.

Please test the ISOs and everything you usually use and rely on, and report back, so we can complete the stable release soon. If no major problems come up, the next iteration will be the stable release, which is scheduled for end of July 2021.

You might have heard about the Freenode IRC drama, and after more than 17 years of #grml on Freenode, it's time to say goodbye.

We decided to move our IRC to irc.oftc.net, so please join us at #grml over there.

Thanks for the many years of great service to Freenode, thanks OFTC for hosting us!

Recently a friend asked me how to automatically start or reconnect to a GNU Screen session directly via OpenSSH’s configuration file. Here’s how to do it:

Add an entry to ~/.ssh/config similar to this one:

Host screen_on_server
    Hostname server.example.org
    RequestTTY yes
    RemoteCommand screen -RD

and then just call ssh screen_on_server and you’ll get connected to an existing screen session if present, otherwise you’ll a new new one.

Should work with tmux, too, but might need commandline different options.

• ga="git annex"
• gap="git add -p"
• amend="git commit --amend"

Hope this might be an inspiration to use these or similar aliases as well.

Long time no see, but there we are - we just released Grml 2020.06 - Ausgehfuahangl!

This Grml release provides fresh software packages from Debian testing (AKA bullseye). As usual it also incorporates current hardware support and fixes known bugs from the previous Grml release.

More information is available in the release notes of Grml 2020.06.

Grab the latest Grml ISO(s) and spread the word!

Thanks everyone, stay healthy and happy Grml-ing!

We are proud to announce the first release candidate of the upcoming version 2020.06, code-named 'Ausgehfuahangl'!

This Grml release provides fresh software packages from Debian testing (AKA bullseye). As usual it also incorporates current hardware support and fixes known bugs from the previous Grml release.

For detailed information about the changes between 2018.12 and 2020.06(-rc1) have a look at the official release announcement.

Please test the ISOs and everything you usually use and rely on, and report back, so we can complete the stable release soon. If no major problems come up, the next iteration will be the stable release, which is scheduled for end of June 2020.

The Debian project hosts a Bug Squashing Party in Salzburg/Austria, taking place from April 5th to April 7th 2019. A Bug Squashing Party is a come-together of developers, contributors and enthusiasts who try to fix as many bugs as possible.

We’ve been invited to join the Bug Squashing Party and since Grml is an official derivative of Debian we’re more than happy to take this opportunity. This is the perfect time and place to meet with fellow Grml and Debian developers, contributors and friends, to make the next Debian- and Grml-Release the best ever! :)

The key facts:

• Location: conova communications GmbH, Karolingerstr. 36A, 5020 Salzburg, Austria
• Date: 2019-04-05 (Friday) until 2019-04-07 (Sunday)

If you want to join us please visit wiki.debian.org/BSP/2019/04/Salzburg for further information. If you’ve any questions feel free to reach out to us.

PS: For the ones of you who can’t join us in Salzburg, feel free to join us during the Bug Squashing Party on IRC (#grml on irc.freenode.org)!

So we did it again - we just released Grml 2018.12 - Gnackwatschn!

This Grml release provides fresh software packages from Debian testing, what's going to be released as stable release Debian/buster in 2019. As usual it also incorporates current hardware support and fixes known bugs from previous Grml releases.

More information is available in the release notes of Grml 2018.12.

Grab the latest Grml ISO(s) and spread the word!

Thanks everyone, happy new year and happy Grml-ing!

We are proud to announce the first release candidate of the upcoming version 2018.12, code-named 'Gnackwatschn'!

This Grml release provides fresh software packages from Debian testing (AKA buster). As usual it also incorporates current hardware support and fixes known bugs from the previous Grml release.

For detailed information about the changes between 2017.05 and 2018.12(-rc1) have a look at the official release announcement.

Please test the ISOs and everything you usually use and rely on, and report back, so we can complete the stable release soon. If no major problems come up, the next iteration will be the stable release, which is scheduled for end of December 2018.

The “properly quote eMail messages and on Usenet” documentation is hosted on a server that appears to not get too much care at the moment. I’ve dug out workable versions:

• current all-in-one-page German version (aktuelle alles-auf-einer-Seite-Fassung auf Deutsch), with a short link on http://deb.li/quote
• last archived Dutch version (laatste versie in het Nederlands), the server is currently unreachable, but the Wayback Machine still has it
• current English translation, still workable

The original link, with its http://learn.to/quote/ redirection, which contained the links to the translations into Dutch and English, unfortunately no longer works.

I’m asking everyone to please honour these guidelines when posting in Usenet and responding to eMail messages, as not doing so is an insult to all the (multiple, in the case of Usenet and mailing lists) readers / recipients of your messages. Even if you have to spend a little time trimming the quote, it’s much less than the time spent by all readers trying to figure out a TOFU (reply over fullquote) message.

Ich bitte jeden darum, sich bitte beim Posten im Usenet und Verfassen von eMails sich an diese Richtilinien zu halten; dies nicht zu tun ist ein Affront wider alle (im Falle von Usenet und Mailinglisten viele) Leser bzw. Empfänger eurer Nachrichten. Selbst wenn man zum Kürzen des Zitats ein bißchen Zeit aufwenden muß ist das immer noch deutlich weniger als die Mühe, die jeder einzelne Leser aufwenden muß, herauszufinden, was mit einer als TOFU (Text oben, Vollzitat unten) geschriebenen eMail gemeint ist.

Mag ik iederéén verzoeken, postings in het Usenet en mailtjes volgens deze regels te schrijven? Als het niet te doen is vies tegen alle ontvanger’s en moeilijk om te lezen. Zelfs als je een beetje tijd nodig heb om het oorspronkelijke deel te korten is het nog steeds minder dan de moeite van alleman, om een TOFU (antwoord boven, fullquote beneden) boodschap proberen te begrepen.

I’m currently working on an mksh(1) and bc(1) script that takes a pitch standard (e.g. “A₄ = 440 Hz” or “C₄ = 256 Hz”) and a config file describing a temperament (e.g. the usual equal temperament, or Pythagorean untempered pure fifths (with the wolf), or “just” intonation, Werckmeister Ⅲ, Vallotti or Bach/Lehman 1722 (to name a few; these are all temperaments that handle enharmonics the same or, for Pythagorean in out case, ignore the fact they’re unplayable). Temperaments are rule-based, like in ttuner. Well, I’m not quite there yet, but I’m already able to display the value for MuseScore to adjust its pitch standard (it can only take A₄-based values), a frequency table, and a list and table of cent deltas (useful for using or comparing with other tuners). Of course, right now, the cent deltas are all 0 because, well, they are equal temperament against equal temperament (as baseline), but I can calculate that with arbitrary and very high precision!

For outputting, I wanted to make the tables align nicely; column(1), which I normally use, was out because it always left-aligns, so I used string padding in Korn Shell — except I’m also a Unicode BMP fan, so I had F♯ and B♭ in my table headings, which were for some reason correctly right-aligned (for when the table values were integers) but not padded right when aligning with the decimal dot. So I worked around it, but also investigated.

Turns out that the desired length was used as second snprintf(3) argument, instead of, as in the right-align case, the buffer size. This worked only until multibyte characters happened. A fun bug, which only took about three minutes to find, and is covered by a new check in the testsuite even. Thought I’d share.

Feedback on and improvements for the tuner, once it’ll be done, are, of course, also welcome. I plan to port the algorithm (once I’ve got it down in a programming language I know well) to QML for inclusion in the tuner MuseScore plugin, even. Check here, for now, for my work in progress… it’s quite big already despite doing basically nothing. Foundation laid (or so…).

Today I’ve released another new CVS snapshot of the FixedMisc [MirOS] font; as usual, the tarball contains the font in BDF form, with no conflict with the system Fixed [Misc] font; sources for use (compilation, editing) with bdfctool(1) are in CVS.

Giacomo Tesio referenced mksh(1) in his annual Jehanne report and provided a guest post (dated 2018-01-09, sorry for posting it this late only) for us on his journey on porting mksh to Jehanne, his Plan 9 derivative operating system. Read on for his story!

(read more…)

paste is one of those tools nobody uses [1]. It puts two file side by side, line by line.

One application for this came up today where some tool was called for several files at once and would spit out one line by file, but unfortunately not including the filename.

$ paste <(ls *.rpm) <(ls *.rpm | xargs -r rpm -q --queryformat '%{name} \n' -p)

[1] See "J" in The ABCs of Unix

[PS: I meant to blog this in 2011, but apparently never committed the file...]

Short Review of the Tex Yoda II

Pro

• Trackpoint
• Cherry MX Switches
• Compact but heavy alumium case
• Backlight (optional)
• USB C connector and USB A to C cable with angled USB C plug
• All three types of Thinkpad Trackpoint caps included
• Configurable layout with nice web-based configurator (might be opensourced in the future)
• Fn+Trackpoint = scrolling (not further configurable, though)
• Case not clipped, but screwed
• Backlight brightness and Trackpoint speed configurable via key bindings (usually Fn and some other key)
• Default Fn keybindings as side printed and backlit labels
• Nice packaging

Contra

• It’s only a 60% Keyboard (I prefer TKL) and the two common top rows are merged into one, switched with the Fn key.
• Cursor keys by default (and labeled) on the right side (mapped to Fn + WASD) — maybe good for games, but not for me.
• ~ on Fn-Shift-Esc
• Occassionally backlight flickering (low frequency)
• Pulsed LED light effect (i.e. high frequency flickering) on all but the lowest brightness level
• Trackpoint is very sensitive even in the slowest setting — use Fn+Q and Fn+E to adjust the trackpoint speed (“tps”)
• No manual included or (obviously) downloadable.
• Only the DIP switches 1-3 and 6 are documented, 4 and 5 are not. (Thanks gismo for the question about them!)
• No more included USB hub like the Tex Yoda I had or the HHKB Lite 2 (USB 1.1 only) has.

My Modifications So Far

Layout Modifications Via The Web-Based Yoda 2 Configurator

• Right Control and Menu key are Right and Left cursors keys
• Fn+Enter and Fn+Shift are Up and Down cursor keys
• Right Windows key is the Compose key (done in software via xmodmap)
• Middle mouse button is of course a middle click (not Fn as with the default layout).

Other Modifications

• Clear dampening o-rings (clear, 50A) under each key cap for a more silent typing experience
• Braided USB cable

Next Swiss Mechanical Keyboard Meetup

On Sunday, the 18th of February 2018, the 4th Swiss Mechanical Keyboard Meetup will happen, this time at ETH Zurich, building CAB, room H52. I’ll be there with at least my Tex Yoda II and my vintage Cherry G80-2100.

Why I ordered a $300 keyboard

(JFTR: It was actually USD $299 plus shipping from the US to Europe and customs fee in Switzerland. Can’t exactly find out how much of shipping and customs fee were actually for that one keyboard, because I ordered several items at once. It’s complicated…)

I always was and still are a big fan of Trackpoints as common on IBM and Lenovo Thinkapds as well as a few other laptop manufactures.

For a while I just used Thinkpads as my private everyday computer, first a Thinkpad T61, later a Thinkpad X240. At some point I also wanted a keyboard with Trackpoint on my workstation at work. So I ordered a Lenovo Thinkpad USB Keyboard with Trackpoint. Then I decided that I want a permanent workstation at home again and ordered two more such keyboards: One for the workstation at home, one for my Debian GNU/kFreeBSD running ASUS EeeBox (not affected by Meltdown or Spectre, yay! :-) which I often took with me to staff Debian booths at events. There, a compact keyboard with a built-in pointing device was perfect.

Then I met the guys from the Swiss Mechanical Keyboard Meetup at their 3rd meetup (pictures) and knew: I need a mechanical keyboard with Trackpoint.

IBM built one Model M with Trackpoint, the M13, but they’re hard to get. For example, ClickyKeyboards sells them, but doesn’t publish the price tag. :-/ Additionally, back then there were only two mouse buttons usual and I really need the third mouse button for unix-style pasting.

Then there’s the Unicomp Endura Pro, the legit successor of the IBM Model M13, but it’s only available with an IMHO very ugly color combination: light grey key caps in a black case. And they want approximately 50% of the price as shipping costs (to Europe). Additionally it didn’t have some other nice keyboard features I started to love: Narrow bezels are nice and keyboards with backlight (like the Thinkpad X240 ff. has) have their advantages, too. So … no.

Soon I found, what I was looking for: The Tex Yoda, a nice, modern and quite compact mechanical keyboard with Trackpoint. Unfortunately it is sold out since quite some years ago and more then 5000 people on Massdrop were waiting for its reintroduction.

And then the unexpected happened: The Tex Yoda II has been announced. I knew, I had to get one. From then on the main question was when and where will it be available. To my surprise it was not on Massdrop but at a rather normal dealer, at MechanicalKeyboards.com.

At that time a friend heard me talking of mechanical keyboards and of being unsure about which keyboard switches I should order. He offered to lend me his KBTalking ONI TKL (Ten Key Less) keyboard with Cherry MX Brown switches for a while. Which was great, because from theory, MX Brown switches were likely the most fitting ones for me. He also gave me two other non-functional keyboards with other Cherry MX switch colors (variants) for comparision. As a another keyboard to compare I had my programmable Cherry G80-2100 from the early ’90s with vintage Cherry MX Black switches. Another keyboard to compare with is my Happy Hacking Keyboard (HHKB) Lite 2 (PD-KB200B/U) which I got as a gift a few years ago. While the HHKB once was a status symbol amongst hackers and system administrators, the old models (like this one) only had membrane type keyboard switches. (They nevertheless still seem to get built, but only sold in Japan.)

I noticed that I was quickly able to type faster with the Cherry MX Brown switches and the TKL layout than with the classic Thinkpad layout and its rubber dome switches or with the HHKB. So two things became clear:

• At least for now I want Cherry MX Brown switches.
• I want a TKL (ten key less) layout, i.e. one without the number block but with the cursor block. As with the Lenovo Thinkpad USB Keyboards and the HHKB, I really like the cursor keys being in the easy to reach lower right corner. The number pad is just in the way to have that.

Unfortunately the Tex Yoda II was without that cursor block. But since it otherwise fitted perfectly into my wishlist (Trackpoint, Cherry MX Brown switches available, Backlight, narrow bezels, heavy weight), I had to buy one once available.

So in early December 2017, I ordered a Tex Yoda II White Backlit Mechanical Keyboard (Brown Cherry MX) at MechanicalKeyboards.com.

Because I was nevertheless keen on a TKL-sized keyboard I also ordered a Deck Francium Pro White LED Backlit PBT Mechanical Keyboard (Brown Cherry MX) which has an ugly font on the key caps, but was available for a reduced price at that time, and the controller got quite good reviews. And there was that very nice Tai-Hao 104 Key PBT Double Shot Keycap Set - Orange and Black, so the font issue was quickly solved with keycaps in my favourite colour: orange. :-)

The package arrived in early January. The aluminum case of the Tex Yoda II was even nicer than I thought. Unfortunately they’ve sent me a Deck Hassium full-size keyboard instead of the wanted TKL-sized Deck Francium. But the support of MechanicalKeyboards.com was very helpful and I assume I can get the keyboard exchanged at no cost.

I’ll not respond, much, until next Monday. We have FrOSCon.

Considering I’ve become the de-facto upstream of cvs(GNU) even if not yet formally the de-iure upstream maintainer, fixing this bug obviously falls to me — not quite the way I had planned passing this evening after coming home from work and a decent and, worse, very filling meal at the local Croatian restaurant. But, so’s life.

The problem here is basically that CVS invokes ssh(1) (well, rsh originally…) but doesn’t add the argument separator “--” before the (user-provided) hostname, which when starting with a hyphen-minus will be interpreted by ssh as an argument. (Apparently the other VCSes also had additional vulnerabilities such as not properly escaping semicoloi or pipes from the shell or unescaping percent-escaped fun characters, but that doesn’t affect us.)

The obvious fix and the one I implemented first is to simply add the dashes. This will also be backported to Debian {,{,old}old}stable-security.

Then I looked at other VCSes out of which only one did this, but they all added extra paranoia hostname checks (some of them passing invalid hostnames, such as those with underscores in them). OK, I thought, then also let’s add extra checks to CVS’ repository reference handling. This will end up in Debian sid and MirBSD, pending passing the regression tests of course… hah, while writing this article I had to fixup because a test failed. Anyway, it’s not strictly necessary AFAICT to fix the issue.

Update, about 2⅕ hours past midnight (the testsuite runs for several hours): of course, the “sanity” testsuite (which itself is rather insane…) also needs adjustments, plus a bonus fix (for something that got broken when the recent allow-root-regex patch was merged and got fixed in the same go to…night).

tl;dr: a fix will end up in Debian *stable-security and can be taken out of my mail to the bugreport; another few changes for robustness are being tested and then added to both MirBSD and Debian sid. The impact is likely small, as it’s hard to get a user (if you find one, in the first place) to use a crafted CVSROOT string, which is easy to spot as well.

Update, Monday: apparently someone took care of the DSA and DLA yesterday after ACCEPTing the uploads — thanks, I was outside during the day.

Update 2017-08-25: It was noted that ssh(1) does not parse its command line correctly, and therefore the patch above might not be enough in the general case. However, I still think it’s good enough for CVS because it constructs its command line in a way that doesn’t let users exploit that bug.

mksh R56 was released with experimental fixes for the “history no longer persisted when HISTFILE near-full” and interactive shell cannot wait on coprocess by PID issues (I hope they do not introduce any regressioins) and otherwise as a bugfix release. You might wish to know the $EDITOR selection mechanism in dot.mkshrc changed. Some more alias characters are allowed again, and POSIX character classes (for ASCII, and EBCDIC, only) appeared by popular vote.

mksh now has a FAQ; enjoy. Do feel free to contribute (answers, too, of course).

The jupp text editor has also received a new release; asides from being much smaller, and updated (mksh too, btw) to Unicode 10, and some segfault fixes, it features falling back to using /dev/tty if stdin or stdout is not a terminal (for use on GNU with find | xargs jupp, since they don’t have our xargs(1) -o option yet), a new command to exit nonzero (sometimes, utilities invoking the generic visual editor need this), and “presentation mode”.

Presentation mode, crediting Natureshadow, is basically putting your slides as (UTF-8, with fancy stuff inside) plaintext files into one directory, with sorting names (so e.g. zero-padded slide numbers as filenames), presenting them with jupp * in a fullscreen xterm. You’d hit F6 to switch to one-file view first, then present by using F8 to go forward (F7 to go backward), and, for demonstrations, F9 to pipe the entire slide through an external command (could be just “sh”) offering the previous one as default. Simple yet powerful; I imagine Sven Guckes would love it, were he not such a vim user.

The new release is offered as source tarball (as usual) and in distribution packages, but also, again, a Win32 version as PKZIP archive (right-click on setup.inf and hit I̲nstall to install it). Note that this comes with its own (thankfully local) version of the Cygwin32 library (compatible down to Windows 95, apparently), so if you have Cygwin installed yourself you’re better off compiling it there and using your own version instead.

I’ve also released a new DOS version of 2.8 with no code patches but an updated jupprc; the binary (self-extracting LHarc archive) this time comes with all resource files, not just jupp’s.

Today, the jupprc drop-in file for JOE 3.7 got a matching update (and some fixes for bugs discovered during that) and I added a new one for JOE 4.4 (the former being in Debian wheezy, the latter in jessie, stretch and buster/sid). It’s a bit rudimentary (the new shell window functionality is absent) but, mostly, gives the desired jupp feeling, more so than just using stock jstar would.

• source tarballs
• Win32 binaries
• DOS binaries
• drop-in jupprc for JOE 2.8, or to update jupp 2.8
• drop-in jupprc for JOE 3.7
• drop-in jupprc for JOE 4.4

CVS’ ability to commit to multiple branches of a file at the same time, therefore grouping the commit (by commitid at least, unsure if cvsps et al. can be persuaded to recognise it). If you don’t know what cvs(GNU) is: it is a proper (although not distributed) version control system and the best for centralised tasks. (For decentral tasks, abusing git as pseudo-VCS has won by popularity vote; take this as a comparison.)

If desired, I can make these new versions available in my “WTF” APT repository on request. (Debian buster/sid users: please change “https” to “http” there, the site is only available with TLSv1.0 as it doesn’t require bank-level security.)

I’d welcome it very much if people using an OS which does not yet carry either to package it there. Message me when one more is added, too ☺

In unrelated news I uploaded MuseScore 2.1 to Debian unstable, mostly because the maintainers are busy (though I could comaintain it if needed, I’d just need help with the C++ and CMake details). Bonus side effect is that I can now build 2.2~ test versions with patches of mine added I plan to produce to fix some issues (and submit upstream) ☻

(read more…)

So we did it again - we just released Grml 2017.05 - Freedatensuppe!

This Grml release provides fresh software packages from Debian testing, what's going to be released as stable release Debian/stretch in just a few more days. As usual it also incorporates current hardware support and fixes known bugs from previous Grml releases.

More information is available in the release notes of Grml 2017.05.

Grab the latest Grml ISO(s) and spread the word!

Thanks everyone and happy grml-ing!

We are proud to announce the first release candidate of the upcoming version 2017.05, code-named 'Freedatensuppe'!

This Grml release provides fresh software packages from Debian testing (AKA stretch). As usual it also incorporates current hardware support and fixes known bugs from the previous Grml release.

For detailed information about the changes between 2014.11 and 2017.05(-rc1) have a look at the official release announcement.

Please test the ISOs and everything you usually use and rely on, and report back, so we can complete the stable release soon. If no major problems come up, the next iteration will be the stable release, which is scheduled for end of May (2017 :)).

For sharing multiple files, it is often convenient to pack them into an archive, and the most widely supported format to do so is probably ZIP. Under *nix, you can archive a directory with Info-ZIP:

% zip -r something.zip something/

(When you have several files, it is recommended to archive them in a directory, to avoid cluttering the directory where people will extract them.)

Unsupported ZIP archive

Unfortunately, while we would expect ZIP files to be widely supported, I found out that this is not always the case, and I had many recipients failing to open them under operating systems such as iOS.

Avoid extra fields

That issue seems to be linked to the usage of extra file attributes, that are enabled by default, in order to store Unix file metadata. The field designed to store such extra attributes was designed from the beginning so each implementation can take into account attributes it supports and ignore any other ones, but some buggy ZIP implementation appear not to function at all with them.

Therefore, unless you actually need to preserve Unix file metadata, you should avoid using extra fields. With Info-ZIP, you would have to add the option -X:

% zip -rX something.zip something/

Someone from the FSF’s licencing department posted an official-looking thing saying they don’t believe GitHub’s new ToS to be problematic with copyleft. Well, my lawyer (not my personal one, nor for The MirOS Project, but related to another association, informally) does agree with my reading of the new ToS, and I can point out at least a clause in the GPLv1 (I really don’t have time right now) which says contrary (but does this mean the FSF generally waives the restrictions of the GPL for anything on GitHub?). I’ll eMail GitHub Legal directly and will try to continue getting this fixed (as soon as I have enough time for it) as I’ll otherwise be forced to force GitHub to remove stuff from me (but with someone else as original author) under GPL, such as… tinyirc and e3.

My dbconfig-common Debian packaging example got a rather hefty upgrade because dbconfig-common (unlike any other DB schema framework I know of) doesn’t apply the upgrades on a fresh install (and doesn’t automatically put the upgrades into a transaction either) but only upgrades between Debian package versions (which can be funny with backports, but AFAICT that part is handled correctly). I now append the upgrades to the initial-version-as-seen-in-the-source to generate the initial-version-as-shipped-in-the-binary-package (optionally, only if it’s named .in) removing all transaction stuff from the upgrade files and wrapping the whole shit in BEGIN; and COMMIT; after merging. (This should at least not break nōn-PostgreSQL databases and… well, database-like-ish things I cannot test for obvious (SQLite is illegal, at least in Germany, but potentially worldwide, and then PostgreSQL is the only remaining Open Source database left ;) reasons.)

Update: Yes, this does mean that maintainers of databases and webservers should send me patches to make this work with not-PostgreSQL (new install/name.in, upgrade files) and not-Apache-2.2/2.4 (new debian/*/*.conf snippets) to make this packaging example even more generally usable.

Natureshadow already forked this and made a Python/Flask package from it, so I’ll prod him to provide a similarily versatile hello-python-world example package.

Since I use this as base for other PHP packages like SimKolab, I’ve updated my packaging example with:

• PHP 7 support (untested, as I need libapache2-mod-php5)
• tons more utility code for you to use
• a class autoloader, with example (build time, for now)
• (at build time) running a PHPUnit testsuite (unless nocheck)

The old features (Apache 2.2 and 2.4 support, dbconfig-common, etc.) are, of course, still there. Support for other webservers could be contributed by you, and I could extend the autoloader to work at runtime (using dpkg triggers) to include dependencies as packaged in other Debian packages. See, nobody needs “composer”! ☻

Feel free to check it out, play around with it, install it, test it, send me improvement patches and feature requests, etc. — it’s here with a mirror at GitHub (since I wrote it myself and the licence is permissive enough anyway).

This posting and the code behind it are sponsored by my employer ⮡ tarent.

Please use the correct (perma)link to bookmark this article, not the page listing all wlog entries of the last decade. Thank you.</update>

Some updates inline and at the bottom.

The new Terms of Service of GitHub became effective today, which is quite problematic — there was a review phase, but my reviews pointing out the problems were not answered, and, while the language is somewhat changed from the draft, they became effective immediately.

Now, the new ToS are not so bad that one immediately must stop using their service for disagreement, but it’s important that certain content may no longer legally be pushed to GitHub. I’ll try to explain which is affected, and why.

I’m mostly working my way backwards through section D, as that’s where the problems I identified lie, and because this is from easier to harder.

Note that using a private repository does not help, as the same terms apply.

Anything requiring attribution (e.g. CC-BY, but also BSD, …)

Section D.7 requires the person uploading content to waive any and all attribution rights. Ostensibly “to allow basic functions like search to work”, which I can even believe, but, for a work the uploader did not create completely by themselves, they can’t grant this licence.

The CC licences are notably bad because they don’t permit sublicencing, but even so, anything requiring attribution can, in almost all cases, not “written or otherwise, created or uploaded by our Users”. This is fact, and the exceptions are few.

Anything putting conditions on the right to “use, display and perform” the work and, worse, “reproduce” (all Copyleft)

Section D.5 requires the uploader to grant all other GitHub users…

• the right to “use, display and perform” the work (with no further restrictions attached to it) — while this (likely — I didn’t check) does not exclude the GPL, many others (I believe CC-*-SA) are affected, and…
• the right to “reproduce your Content solely on GitHub as permitted through GitHub's functionality”, with no further restructions attached; this is a killer for, I believe, any and all licences falling into the “copyleft” category.

Note that section D.4 is similar, but granting the licence to GitHub (and their successors); while this is worded much more friendly than in the draft, this fact only makes it harder to see if it affects works in a similar way. But that doesn’t matter since D.5 is clear enough. (This doesn’t mean it’s not a problem, just that I don’t want to go there and analyse D.4 as D.5 points out the same problems but is easier.)

This means that any and all content under copyleft licences is also no longer welcome on GitHub.

Anything requiring integrity of the author’s source (e.g. LPPL)

Some licences are famous for requiring people to keep the original intact while permitting patches to be piled on top; this is actually permissible for Open Source, even though annoying, and the most common LaTeX licence is rather close to that. Section D.3 says any (partial) content can be removed — though keeping a PKZIP archive of the original is a likely workaround.

Affected licences

Anything copyleft (GPL, AGPL, LGPL, CC-*-SA) or requiring attribution (CC-BY-*, but also 4-clause BSD, Apache 2 with NOTICE text file, …) are affected. BSD-style licences without advertising clause (MIT/Expat, MirOS, etc.) are probably not affected… if GitHub doesn’t go too far and dissociates excerpts from their context and legal info, but then nobody would be able to distribute it, so that’d be useless.

But what if I just fork something under such a licence?

Only “continuing to use GitHub” constitutes accepting the new terms. This means that repositories from people who last used GitHub before March 2017 are excluded.

Even then, the new terms likely only apply to content uploaded in March 2017 or later (note that git commit dates are unreliable, you have to actually check whether the contribution dates March 2017 or later).

And then, most people are likely unaware of the new terms. If they upload content they themselves don’t have the appropriate rights (waivers to attribution and copyleft/share-alike clauses), it’s plain illegal and also makes your upload of them or a derivate thereof no more legal.

Granted, people who, in full knowledge of the new ToS, share any “User-Generated Content” with GitHub on or after 1ˢᵗ March, 2017, and actually have the appropriate rights to do that, can do that; and if you encounter such a repository, you can fork, modify and upload that iff you also waive attribution and copyleft/share-alike rights for your portion of the upload. But — especially in the beginning — these will be few and far between (even more so taking into account that GitHub is, legally spoken, a mess, and they don’t even care about hosting only OSS / Free works).

Conclusion (Fazit)

I’ll be starting to remove any such content of mine, such as the source code mirrors of jupp, which is under the GNU GPLv1, now and will be requesting people who forked such repositories on GitHub to also remove them. This is not something I like to do but something I am required to do in order to comply with the licence granted to me by my upstream. Anything you’ve found contributed by me in the meantime is up for review; ping me if I forgot something. (mksh is likely safe, even if I hereby remind you that the attribution requirement of the BSD-style licences still applies outside of GitHub.)

(Pet peeve: why can’t I “adopt a licence” with British spelling? They seem to require oversea barbarian spelling.)

The others

Atlassian Bitbucket has similar terms (even worse actually; I looked at them to see whether I could mirror mksh there, and turns out, I can’t if I don’t want to lose most of what few rights I retain when publishing under a permissive licence). Gitlab seems to not have such, but requires you to indemnify them… YMMV. I think I’ll self-host the removed content.

And now?

I’m in contact with someone from GitHub Legal (not explicitly in the official capacity though) and will try to explain the sheer magnitude of the problem and ways to solve this (leaving the technical issues to technical solutions and requiring legal solutions only where strictly necessary), but for now, the ToS are enacted (another point of my criticism of this move) and thus, the aforementioned works must go off GitHub right now.

That’s not to say they may not come back later once this all has been addressed, if it will be addressed to allow that. The new ToS do have some good; for example, the old ToS said “you allow every GitHub user to fork your repositories” without ever specifying what that means. It’s just that the people over at GitHub need to understand that, both legally and technically¹, any and all OSS licences² grant enough to run a hosting platform already³, and separate explicit grants are only needed if a repository contains content not under an OSI/OKFN/Copyfree/FSF/DFSG-free licence. I have been told that “these are important issues” and been thanked for my feedback; we’ll see what comes from this.

① maybe with a little more effort on the coders’ side³

② All licences on one of those lists or conformant to the DFSG, OSD or OKD should do⁴.

③ e.g. when displaying search results, add a note “this is an excerpt, click HERE to get to the original work in its context, with licence and attribution” where “HERE” is a backlink to the file in the repository

④ It is understood those organisations never un-approve any licence that rightfully conforms to those definitions (also in cases like a grant saying “just use any OSS² licence” which is occasionally used)

Update: In the meantime, joeyh has written not one but two insightful articles (although I disagree in some details; the new licence is only to GitHub users (D.5) and GitHub (D.4) and only within their system, so, while uploaders would violate the ToS (they cannot grant the licence) and (probably) the upstream-granted copyleft licence, this would not mean that everyone else wasn’t bound by the copyleft licence in, well, enough cases to count (yes it’s possible to construct situations in which this hurts the copyleft fraction, but no, they’re nowhere near 100%).

This article might be perceived as a blatant ripoff of this Linux kernel document, but, on the contrary, it’s intended as add-on, showing how to do a subtree merge (the multi-project merge strategy that’s actually doable in a heterogenous group of developers, as opposed to subprojects, which many just can’t wrap their heads around) with contemporary git (“stupid content tracker”). Furthermore, the commands are reformatted to be easier to copy/paste.

To summarise: you’re on the top level of a checkout of the project into which the “other” project (Bproject) is to be merged. We wish to merge the top level of Bproject’s “master” branch as (newly created) subdirectory “dir-B” under the current project’s top level.

	$ git remote add --no-tags -f Bproject /path/to/B/.git
	$ git merge -s ours --allow-unrelated-histories --no-commit Bproject/master
	$ git read-tree -u --prefix=dir-B/ Bproject/master
	$ git commit -m 'Merge B project as our subdirectory dir-B'

	Later updates are easy:
	$ git pull -s subtree Bproject master
 

(mind the trailing slash after dir-B/ on the read-tree command!)

Besides reformatting, the use of --allow-unrelated-histories recently became necessary. --no-tags is also usually what you want, because tags are not namespaced like branches.

Another command you might find relevant is how to clean up orphaned remote branches:

	$ for x in $(git remote); do git remote prune "$x"; done
 

This command locally deletes all remote branches (those named “origin/foo”) that have been deleted on the remote side.

Update: Natureshadow wishes you to know that there is such a command as git subtree which can do similar things to the subtree merge strategy explained above, and several more related things. It does, however, need the præfix on every subsequent pull.

No man pages, but almost

The AWS Command Line Interface, which is available in Debian, provides no man page. Instead, that tool has an integrated help system, which allows you to run commands such as aws rds help, that, for what I have seen, generates some reStructuredText, then converts it to a man page in troff format, then calls troff to convert it to text with basic formatting, and eventually passes it to a pager. Since this is close to what man does, the result looks like a degraded man page, with some features missing such as the adaptation to the terminal width.

Well, this is better than nothing, and better than what many under-documented tools can offer, but for several reasons, it still sucks: most importantly, it does not respect administrators' habits and it does not integrate with the system man database. You it does not allow you to use commands such as apropos, and you will get no man page name auto-completion from your shell since there is no man page.

Generate the man pages

Now, since the integrated help system does generate a man page internally, we can hack it to output it, and save it to a file:

Description: Enable a mode to generate troff man pages
 The awscli help system internally uses man pages, but only to convert
 them to text and show them with the pager. This patch enables a mode
 that prints the troff code so the user can save the man page.
 .
 To use that mode, run the help commands with an environment variable
 OUTPUT set to 'troff', for instance:
     OUTPUT='troff' aws rds help
Forwarded: no
Author: Tanguy Ortolo <tanguy+debian@ortolo.eu>
Last-Update: 2016-11-22

Index: /usr/lib/python3/dist-packages/awscli/help.py
===================================================================
--- /usr/lib/python3/dist-packages/awscli/help.py       2016-11-21 12:14:22.236254730 +0100
+++ /usr/lib/python3/dist-packages/awscli/help.py       2016-11-21 12:14:22.236254730 +0100
@@ -49,6 +49,8 @@
     Return the appropriate HelpRenderer implementation for the
     current platform.
     """
+    if 'OUTPUT' in os.environ and os.environ['OUTPUT'] == 'troff':
+        return TroffHelpRenderer()
     if platform.system() == 'Windows':
         return WindowsHelpRenderer()
     else:
@@ -97,6 +99,15 @@
         return contents


+class TroffHelpRenderer(object):
+    """
+    Render help content as troff code.
+    """
+
+    def render(self, contents):
+        sys.stdout.buffer.write(publish_string(contents, writer=manpage.Writer()))
+
+
 class PosixHelpRenderer(PagingHelpRenderer):
     """
     Render help content on a Posix-like system.  This includes

This patch must be applied from the root directory with patch -p0, otherwise GNU patch will not accept to work on files with absolute names.

With that patch, you can run help commands with an environment variable OUTPUT='troff' to get the man page to use it as you like, for instance:

% OUTPUT='troff' aws rds help > aws_rds.1
% man -lt aws_rds.1 | lp

Generate all the man pages

Now that we are able to generate the man page of any aws command, all we need to generate all of them is a list of all the available commands. This is not that easy, because the commands are somehow derived from functions provided by a Python library named botocore, which are derived from a bunch of configuration files, and some of them are added, removed or renamed. Anyway, I have been able to write a Python script that does that, but it includes a static list of these modifications:

#! /usr/bin/python3

import subprocess
import awscli.clidriver


def write_manpage(command):
    manpage = open('%s.1' % '_'.join(command), 'w')
    command.append('help')
    process = subprocess.Popen(command,
            env={'OUTPUT': 'troff'},
            stdout=manpage)
    process.wait()
    manpage.close()


driver = awscli.clidriver.CLIDriver()
command_table = driver._get_command_table()

renamed_commands = \
    {
        'config': 'configservice',
        'codedeploy': 'deploy',
        's3': 's3api'
    }
added_commands = \
    {
        's3': ['cp', 'ls', 'mb', 'mv', 'presign', 'rb', 'rm', 'sync',
               'website']
    }
removed_subcommands = \
    {
        'ses': ['delete-verified-email-address',
                'list-verified-email-addresses',
                'verify-email-address'],
        'ec2': ['import-instance', 'import-volume'],
        'emr': ['run-job-flow', 'describe-job-flows',
                'add-job-flow-steps', 'terminate-job-flows',
                'list-bootstrap-actions', 'list-instance-groups',
                'set-termination-protection',
                'set-visible-to-all-users'],
        'rds': ['modify-option-group']
    }
added_subcommands = \
    {
        'rds': ['add-option-to-option-group',
                'remove-option-from-option-group']
    }

# Build a dictionary of real commands, including renames, additions and
# removals.
real_commands = {}
for command in command_table:
    subcommands = []
    subcommand_table = command_table[command]._get_command_table()
    for subcommand in subcommand_table:
        # Skip removed subcommands
        if command in removed_subcommands \
                and subcommand in removed_subcommands[command]:
            continue
        subcommands.append(subcommand)
    # Add added subcommands
    if command in added_subcommands:
        for subcommand in added_subcommands[command]:
            subcommands.append(subcommand)
    # Directly add non-renamed commands
    if command not in renamed_commands:
        real_commands[command] = subcommands
    # Add renamed commands
    else:
        real_commands[renamed_commands[command]] = subcommands
# Add added commands
for command in added_commands:
    real_commands[command] = added_commands[command]

# For each real command and subcommand, generate a manpage
write_manpage(['aws'])
for command in real_commands:
    write_manpage(['aws', command])
    for subcommand in real_commands[command]:
        write_manpage(['aws', command, subcommand])
                         'sync', 'website']}

This script will generate more than 2,000 man page files in the current directory; you will then be able to move them to /usr/local/share/man/man1.

Since this is a lot of man pages, it may be appropriate to concatenate them by major command, for instance all the aws rds together…

cnuke@ spotted something on the internet, and shared. Do read this, including the comments. It’s so true. (My car is 30 years old, I use computers mostly for sirc, lynx and ssh, and I especially do not buy any product that needs to be “online” to work.)

Nice parts of the internet, to offset this, though, do exist. IRC as a way of cheap (affordable), mostly reliant, communication that’s easy enough to do with TELNET.EXE if necessary. Fanfiction; easy proliferation of people’s art (literature, in this case). Fast access to documentation and source code; OpenBSD’s AnonCVS was a first, nowadays almost everything (not Tom Dickey’s projects (lynx, ncurses, xterm, cdk, …), nor GNU bash, though) is on a public version control system repository. (Now people need to learn to not rewrite history, just commit whatever shit they do, to record thought process, not produce the perfect-looking patch.) Livestreams too, I guess, but ever since live365.com went dead due to a USA law change on 2016-01-02, it got bad.

GMane has been down for a day or two, and flakey for a day before that. MidnightBSD’s laffer1 just linked the reason, which made me cry out loud.

GMane is really great, and I rely on the NNTP interface a lot, both posting and especially reading — it gives me the ability to download messages from mailing lists I don’t receive in order to be able to compose replies with (mostly) correct References and In-Reply-To headers. Its web interface, especially the article permalinks, are also extremely helpful.

This is a request for a petition to save GMane. Please, someone, do something! Thanks in advance!

When writing a wrapper script, one often has to process the command line arguments to transform them according to his needs, to change some arguments, to remove or insert some, or perhaps to reorder them.

Naive approach

The naive approach to do that is¹:

# Process arguments, building a new argument list
new_args=""
for arg in "$@"
do
    case "$arg"
    in
        --foobar)
            # Convert --foobar to the new syntax --foo=bar
            new_args="$args --foo=bar"
        ;;
        *)
            # Take other options as they are
            new_args="$args $arg"
        ;;
    esac
done

# Call the actual program
exec program $new_args

This naive approach is simple, but fragile, as it will break on arguments that contain a space. For instance, calling wrapper --foobar "some file" (where some file is a single argument) will result in the call program --foo=bar some file (where some and file are two distinct arguments).

Correct approach

To handle spaces in arguments, we need either:

• to quote them in the new argument list, but that requires escaping possible quotes they contain, which would be error-prone, and implies using external programs such as sed;
• to use an actual list or array, which is a feature of advanced shells such as Bash or Zsh, not standard shell…

… except standard shell does support arrays, or rather, it does support one specific array: the positional parameter list "$@"². This leads to one solution to process arguments in a reliable way, which consists in rebuilding the positional parameter list with the built-in command set --:

# Process arguments, building a new argument list in "$@"
# "$@" will need to be cleared, not right now but on first iteration only
first_iter=1
for arg in "$@"
do
    if [ "$first_iter" -eq 1 ]
    then
        # Clear the argument list
        set --
        first_iter=0
    fi
    case "$arg"
    in
        --foobar) set -- "$@" --foo=bar ;;
        *) set -- "$@" "$arg" ;;
    esac
done

# Call the actual program
exec program "$@"

Notes

1. I you prefer, for arg in "$@" can be simplified to just for arg.↑
2. As a reminder, and contrary to what it looks like, quoted "$@" does not expand to a single field, but to one field per positional parameter. ↑

Let's Encrypt is a certificate authority (CA) that just left beta stage, that provides domain name-validated (DV) X.509 certificates for free and in an automated way: users just have to run a piece of software on their server to get and install a certificate, resulting in a valid TLS setup.

A threat to other certificate authorities

By providing certificates for free and automatically, Let's Encrypt is probably a threat a other CAs, a least for part of their activity. Indeed, for people that are satisfied with DV certificates, there are not many reasons to pay a commercial CA to get certificates in a non-automated way. For the CAcert non-commercial CA, that may mean a slow death, as this is their main activity¹.

For people that want organization-validated (OV) or extended validation (EV) certificates, Let's Encrypt is not suitable, so it will not change anything regarding that.

An opportunity for the most reactive

The entrance of Let's Encrypt is also a significant opportunity for the certificate authorities that will be reactive enough to take advantage of their innovation. Indeed, they introduced automation in both domain name validation and certificate issuance (and revocation), by defining an open protocol that is meant to become an Internet standard. That protocol, named ACME, is not tied to Let's Encrypt and has several free software implementations, so it could be used for the same purpose by commercial CAs.

A certification authority could, for instance:

• ask the customer to provision some pre-paid account;
• manually validate the customer's identity once;
• allow the customer to register using ACME, and associate that registration to his validated identity;
• allow the customer to get organization-validated, or perhaps even extended validation certificates using ACME, and making corresponding debits to his pre-paid account.

Such processes may require or benefit from improvements of the ACME protocol, which is the very reason Internet standards are defined in an open way.

The first certification authority that would implement such a process could gain an advantage over its competitors, as it would greatly simplify getting and renewing certificates. I think even Let's Encrypt people would be happy to see that happen, as it would serve their goal, that is basically to help securing the Internet! Personally, I could buy such a service (assuming it is not restricted to juridical persons, according to a quite common (and detestable) sale discrimination against natural persons²).

Notes

1. CAcert is an unrecognised certificate authority, that provides an identity validation through a web of trust, and issues DV server certificates that do not include the validated identity. Now that Let's Encrypt can issue valid DV certificates, CAcert is no longer relevant for that activity. It also issues personal certificates, that do include the validated identity, and that can be used for encryption (e.g. S/MIME), signing (e.g. code signing) or authentication, which is an activity Let's Encrypt does not compete with.
2. Yes, the Organization field of a certificate is probably not relevant to indicate a physical person's name, but the CommonName field is. Yes, that field is usually abused to store the domain name, but a proper use would be to put the owner's name in the CommonName field, and the domain names in the subjectAltName field.↑

The MirBSD Korn Shell R52c was published today as bugfix-accumulating release of low upto medium importance. Thanks to everyone who helped squashing all those bugs; this includes our bug reporters who always include reproducer testcases; you’re wonderful!

MirCPIO was also resynchronised from OpenBSD, to address the CVE-2015-{1193,1194} test cases, after a downstream (wow there are so many?) reminded us of it; thanks!
This is mostly to prevent extracting ../foo — either directly or from a symlink(7) — from actually ending up being placed in the parent directory. As such the severity is medium-high. And it has a page now — initially just a landing page / stub; will be fleshed out later.

Uploads for both should make their way into Debian very soon (these are the packages mksh and pax). Uploading backports for mksh (jessie and wheezy-sloppy) have been requested by several users, but none of the four(?) DDs asked about sponsoring them even answered at all, and the regular (current) sponsors don’t have experience with bpo, so… SOL ☹

I’ve also tweaked a bug in sed(1), in MirBSD. Unfortunately, this means it now comes with the GNUism -i too: don’t use it, use ed(1) (much nicer anyway) or perlrun(1) -p/-n…

Finally, our PDF manpages now use the PA4 paper size instead of DIN ISO A4, meaning they can be printed without cropping or scaling on both A4 and US-american “letter” paper. And a Бодун from the last announcement: we now use Gentium and Inconsolata as body text and monospace fonts, respectively. (And à propos, the website ought to be more legible due to text justification and better line spacing now.) I managed to hack this up in GNU groff and Ghostscript, thankfully. (LaTeX too) Currently there are PDF manpages for joe (jupp), mksh, and cpio/pax/tar.

And we had Grünkohl today!

Also, new console-setup package in the “WTF” APT repository since upstream managed to do actual work on it (even fixed some bugs). Read its feed if interested, as its news will not be repeated here usually. (That means, subscribe as there won’t be many future reminders in this place.)

The netboot.me service appears to be gone. I’ll not remove our images, but if someone knows what became of it drop us a message (IRC or mailing list will work just fine).

PS: This was originally written on 20160304 but opax refused to be merged in time… Happy Birthday, gecko2! In the meantime, the Street Food festival weekend provided wonderful food at BaseCamp, and headache prevented this from being finished on the fifth.

Update 06.03.2016: The pax changes were too intrusive, so I decided to only backport the fixes OpenBSD did (both those they mentioned and those silently included), well, the applicable parts of them, anyway, instead. There will be a MirCPIO release completely rebased later after all changes are merged and, more importantly, tested. Another release although not set for immediate future should bring a more sensible (and mksh-like) buildsystem for improved portability (and thus some more changes we had to exclude at first).

I’ve also cloned the halfwidth part of the FixedMisc [MirOS] font as FixedMiscHW for use with Qt5 applications, xfonts-base in the “WTF” APT repo. (Debian #809979)

tl;dr: mksh R52c (bugfix-only, low-medium); mircpio 20160306 (security backport; high) with future complete rebase (medium) upstream and in Debian. No mksh backports due to lacking a bpo capable sponsor. New console-setup in “WTF” APT repo, and mksh there as usual. xfonts-base too. netboot.me gone?