Skip to content

Security gotcha with log collection on Azure Kubernetes cluster.

Azure Kubernetes Service provides a nice way to set up Kubernetes
cluster in the cloud. It’s quite practical as AKS is setup by default
with a rich monitoring and reporting environment. By default, all
container logs are collected, CPU and disk data are gathered. πŸ‘

I used AKS to setup a cluster for my first client as a
freelance. Everything was nice until my client asked me why logs
collection was as expensive as the computer resources.πŸ’Έ

Ouch… 🀦

My first reflex was to reduce the amount of logs produced by all our
containers, i.e. start logging at warn level instead of info
. This reduced the amount of logs quite a lot.

But this did not reduce the cost of collecting logs, which looks like
to a be a common issue.

Thanks to the documentation provided by Microsoft, I was able to find
that ContainerInventory data table was responsible of more than 60%
of our logging costs.

What is ContainerInventory ? It’s a facility to monitor the content
of all environment variables from all containers.

Wait… What ? ⚠

Should we be worried about our database credentials which are, legacy
oblige, stored in environment variables ?

Unfortunately, the query shown below confirmed that, yes, we should:
the logs aggregated by Azure contains the database credentials of my

| where TimeGenerated > ago(1h)

Having credentials collected in logs is lackluster from a security
point of view. πŸ™„

And we don’t need it because our environment variables do not change.

Well, it’s now time to fix these issues. πŸ› 

We’re going to:

  1. disable the collection of environment variables in Azure, which
    will reduce cost and plug the potential credential leak
  2. renew all DB credentials, because the previous credentials can be
    considered as compromised (The renewal of our DB passwords is quite
    easy with the script I provided to my client)
  3. pass credentials with files instead of environment variables.

In summary, the service provided by Azure is still nice, but beware of
the default configuration which may contain surprises.

I’m a freelance, available for hire. The site
describes how I can help your projects.

All the best

Read more…

How to run CEWE photo creator on Debian


This post describes how I debug an issue with a proprietary software. I hope this will give you some hint on how to proceed should you face a similar issue. If you’re in a hurry, you can read the TL;DR; version at the end.

After the summer vacations, I’ve decided to offer a photo-book to my mother. I searched for open-source solution but the printed results were lackluster.

Unfortunately, the only possible solution was to use professional service. Some of these services offer a web application to create photo books, but this is painful to use on a slow DSL line. Other services provide a program named CEWE. This proprietary program can be downloaded for Windows, Mac and, lo and behold: Linux !

The download goes quite fast as the downloaded program is a Perl script that does the actual download. I would have preferred a proper Debian package, but at least Linux amd64 is supported.

Once installed, CEWE program is available as an executable and a bunch of shared libraries.

This program works quite well to create a photo album. I won’t go into the details there.

I ran into trouble when trying to connect the application to the service site to order the photo-book: the connection fails with a cryptic message “error code 10000”.

Commercial support was not much help as they insisted that I check my proxy settings. I downloaded again CEWE from another photo service. The new CEWE installation gave me the same error. This showed that the issue was on my side and not on the server’s side.

Given that the error occurred quite fast when trying to connect, I guessed that the connection setup was going south. Since the URL shown in the installation script began with https, I had to check for SSL issues.

I checked certificate issues: curl had no problem connecting to the server mentioned in the Perl script. Wireshark showed that the connection to the server was reset by the server quite fast. I wondered which version of SSL was used by CEWE and ran ldd. To my surprise, I found that ldd did not list libssl. Something weird was going on: SSL was required but CEWE was not linked to libssl…

I used another trick: explore all the menus of the application. This was a good move as I found a checkbox to enable debug report in CEWE in “Options -> paramΓ¨tres -> Service” menu (that may be “options-> parameters -> support” in English CEWE). When set, debug traces are also shown on standard output of CEWE,

And, somewhere in the debug traces, I found:

W (2018-10-30T18:36:37.143) [ 0] ==> QSslSocket: cannot resolve SSLv3_client_method <==

So CEWE was looking for SSL symbols even though ldd did not require libssl…

I guessed that CEWE was using dlopen to open the ssl library. But which file was opened by dlopen ?

Most likely, the guys who wrote the call to dlopen did not want to handle file names with so version (i.e. like, and added code to open directly This file is provided by libssl-dev package, which was already installed on my system.

But wait, CEWE was probably written for Debian stable with an older libssl. I tried libssl1.0-dev.. which conflicts with libssl-dev. Oh well, I can live with that for a while…

And that was it ! With libssl1.0-dev installed, CEWE was able to connect to the photo service web site without problems.

So here’s the TL;DR; version. To run CEWE on Debian, run:

sudo apt install libssl1.0-dev

Last but not least, here are some suggestions for CEWE:

  • use libssl1.1. as libssl1.0 is deprecated and will be removed from Debian
  • place the debug checkbox in “System” widget. This widget was the first I opened when I began troubleshooting. “Service” does not mean much to me. Having this checkbox in both “Service” and “System” widgets would not harm

All the best

[ Edit: I first blamed CEWE for loading libssl in a non-standard way. libssl is actually loaded by QtNetwork. Depending on the way Qt is built, SSL is either disabled (-no-openssl option), loaded by dlopen (default) or loaded with dynamic linking (-openssl-linked). The way Qt is built is CEWE choice. Thanks Uli Schlachter for the heads-up]


New Software::LicenseMoreUtils Perl module


Debian project has rather strict requirements regarding package license. One of these requirements is to provide a copyright file mentioning the license of the files included in a Debian package.

Debian also recommends to provide this copyright information in a machine readable format that contain the whole text of the license(s) or a summary pointing to a pre-defined location on the file system (see this example).

cme and Config::Model::Dpkg::Copyright helps in this task using Software::License module. But this module lacks the following features to properly support the requirements of Debian packaging:

  • license summary
  • support for clause like “GPL version 2 or (at your option) any later version”

Long story short, I’ve written Software::LicenseMoreUtils to provide these missing features. This module is a wrapper around Software::License and has the same API.

Adding license summaries for Debian requires only to update this YAML file.

This modules was written for Debian while keeping other distros in minds. Debian derevatives like Ubuntu or Mind are supported. Adding license summaries for other Linux distribution is straightforward. Please submit a bug or a PR to add support for other distributions.

For more details. please see:


All the best

Shutter, a nice Perl application, may be removed from Debian


Debian is moving away from Gnome2::VFS. This obsolete module will be removed from next release of Debian.

Unfortunately, Shutter, a very nice Gtk2 screenshot application, depends on Gnome::VFS, which means that Shutter will be removed from Debian unless this dependency is removed from shutter. This would be a shame as Shutter is one of the best screenshot tool available on Linux and one of the best looking Perl application. And its popularity is still growing.

Shutter also provides a way to edit screenshots, for instance to mask confidential data. This graphical editor is based on Goo::Canvas which is already gone from Debian.

To be kept on Debian, Shutter must be updated:

  • to use Gnome GIO instead of Gnome2::VFS
  • to use GooCanvas2 instead of Goo::Canvas
  • may be, to be ported to Gtk3 (that’s less urgent)

I’ve done some work to port Shutter to GIO, but I need to face reality: Maintaining cme is taking most of my free time and I don’t have the time to overhaul Shutter.

To view or clone the code, you can either:

See also the bug reports about Shutter problems on Ubuntu bug tracker

I hope this blog will help finding someone to maintain Shutter…

All the best.

cme: some read-write backend features are being deprecated


Config::Model and cme read and write configuration data with a set of “backend” classes, like Config::Model::Backend::IniFile. These classes are managed by Config::Model::BackendMgr.

Well, that’s the simplified view. Actually, the backend manager can handle several different backends to read and write data: read backends are tried until one of them succeeds to read configuration data. And write backend cen be different from the read backend, thus offering the possibility to migrare from one format to another. This feature came at the beginning of the project, back in 2005. This felt like a good idea to let user migrate from one data format to another.

More than 10 years later, this feature has never been used and is handled by a bunch of messy code that hampers further evolution of the backend classes.

So, without further ado, I’m going to deprecate the following features in order to simplify the backend manager:

  • The “custom” backend that can be easily replaced with more standard backend based on Config::Model::Backend::Any. This feature has been deprecated with Config::Model 2.107
  • The possibility to specify more that one backend. Soon, only the first read backend will be taken into account. This will simplify the declaration of backend. The “read_config” parameter, which is currently a list of backend specification, will become a single backend specification. The command cme meta edit will handle the migration of existing model to the new scheme.
  • the “write_config” parameter will be removed.

Unless someone objects, actual removal of these feature will be done in the next few months, after a quite short deprecation period.

All the best

New with cme: a GUI to configure Systemd services


Systemd is powerful, but creating a new service is a task that require creating several files in non obvious location (like /etc/systemd/system or ~/.local/share/systemd/user/). Each file features 2 or more sections (e.g. [Unit], [Service]). And each section supports a lot of parameters.

Creating such Systemd configuration files can be seen as a daunting task for beginners.

cme project aims to make this task easier by providing a GUI that:

  • shows all existing services in a single screen
  • shows all possible sections and parameters with their documentation
  • validates the content of each parameter (if possible)

For instance, on my laptop, the command cme edit systemd-user shows 2 custom services (“free-imap-tunnel@” and “gmail-imap-tunnel@”) with:


The GUI above shows the units for my custom systemd files:

$ ls ~/.config/systemd/user/

and the units installed by Debian packages:

$ find /usr/lib/systemd/user/ -maxdepth 1 \
  '(' -name '*.service' -o -name '*.socket' ')' \
  -printf '%f\n' |sort |head -15

The screenshot above shows the content of the service defined by the following file:

$ cat ~/.config/systemd/user/free-imap-tunnel@.service
Description=Tunnel IMAPS connections to Free with Systemd

# no need to install corkscrew
ExecStart=-/usr/bin/socat -,proxyport=8888

Note that empty parameters are not shown because the “hide empty value” checkbox on top right is enabled.

Likewise, cme is able to edit system files like user files with sudo cme edit systemd:


For more details on how to use the GUI to edit systemd files, please see:

Using a GUI may not be your cup of tea. cme can also be used as a validation tool. Let’s add a parameter with an excessive value to my service:

$ echo "CPUShares = 1000000" >> ~/.local/share/systemd/user/free-imap-tunnel@.service

And check the file with cme:

$ cme check systemd-user 
cme: using Systemd model
loading data
Configuration item 'service:"free-imap-tunnel@" Service CPUShares' has a wrong value:
        value 1000000 > max limit 262144

ok, let’s fix this with cme. The wrong value can either be deleted:

$ cme modify systemd-user 'service:"free-imap-tunnel@" Service CPUShares~'
cme: using Systemd model

Changes applied to systemd-user configuration:
- service:"free-imap-tunnel@" Service CPUShares: '1000000' -> ''

Or modified:

$ cme modify systemd-user 'service:"free-imap-tunnel@" Service CPUShares=2048'
cme: using Systemd model

Changes applied to systemd-user configuration:
- service:"free-imap-tunnel@" Service CPUShares: '1000000' -> '2048'

You can also view the specification of a service using cme:

$ cme dump systemd-user 'service:"free-imap-tunnel@"'---
  CPUShares: 2048
    - '-/usr/bin/socat -,proxyport=8888'
  StandardInput: socket
  Description: Tunnel IMAPS connections to Free with Systemd

The output above matches the content of the service configuration file:

$ cat ~/.local/share/systemd/user/free-imap-tunnel@.service
## This file was written by cme command.
## You can run 'cme edit systemd-user' to modify this file.
## You may also modify the content of this file with your favorite editor.

Description=Tunnel IMAPS connections to Free with Systemd

# no need to install corkscrew now
ExecStart=-/usr/bin/socat -,proxyport=8888

Last but not least, you can use cme shell if you want an interactive ui but cannot use a graphical interface:

$ cme shell systemd-user 
cme: using Systemd model
 >:$ cd service:"free-imap-tunnel@"  Service  
 >: service:"free-imap-tunnel@" Service $ ll -nz Exec*
name      β”‚ type β”‚ value                                                             
ExecStart β”‚ list β”‚ -/usr/bin/socat -,proxyport=8888

 >: service:"free-imap-tunnel@" Service $ ll -nz
name             β”‚ type    β”‚ value                                                             
StartupCPUWeight β”‚ integer β”‚ 100                                                               
CPUShares        β”‚ integer β”‚ 2048                                                              
StartupCPUShares β”‚ integer β”‚ 1024                                                              
StandardInput    β”‚ enum    β”‚ socket                                                            
ExecStart        β”‚ list    β”‚ -/usr/bin/socat -,proxyport=8888

 >: service:"free-imap-tunnel@" Service $ set CPUShares=1024
 >: service:"free-imap-tunnel@" Service $ ll -nz CPUShares 
name      β”‚ type    β”‚ value
CPUShares β”‚ integer β”‚ 1024 

 >: service:"free-imap-tunnel@" Service $ quit

Changes applied to systemd-user configuration:
- service:"free-imap-tunnel@" Service CPUShares: '2048' -> '1024'

write back data before exit ? (Y/n)

Currently, only service, socket and timer units are supported. Please create a bug report on github if you need more.

Installation instructions are detailed at the beginning of Managing Systemd configuration with cme wiki page.

As all softwares, cme probably has bugs. Please report any issue you might have with it.

For more information:

All in all, systemd is quite complex to setup. I hope I made a little bit easier to deal with.

All the best

New dzil command to install author dependencies as Debian packages


Dist::Zilla is a great tool to limit tedious tasks while working on Perl modules. For instance, dzil provides tools like dzil authordeps or dzil listdeps to list dependencies.
This list of Perl modules can then be installed with cpanm:

dzil authordeps --missing | cpanm
dzil listdeps --missing | cpanm

On a Debian system, one may prefer to install Perl modules using Debian packages. Installing build dependencies can be done with apt build-dep, but apt does not handle Dist::Zilla author dependencies.

The new authordebs Dist::Zilla sub-command was wriiten to fill this gap. When run in a directory containing the source of a Perl module that uses Dist::Zilla, you can run dzil installdebs to list the Debian packages required to run the dzil command. You can also run dzil installdebs -install to install author dependencies (using sudo under the hood).


On Debian, authordebs is provided by libdist-zilla-app-command-authordebs-perl

All the best

A survey for developers about application configuration


Markus Raab, the author of Elektra project, has created a survey to get FLOSS developer’s point of view on the configuration of application.

If you are a developer, please fill this survey to help Markus’ work on improving application configuration management. Feeling this survey should take about 15 mns.

Note that the survey will close on July 18th.

The fact that this blog comes 1 month after the beginning of the survey is entirely my fault. Sorry about that…

All the best

An improved Perl API for cme and Config::Model


While hacking on a script to update build dependencies on a Debian package, it occured to me that using Config::Model in a Perl program should be no more complicated than using cme from a shell script. That was an itch that I scratched immediately.

Fast forward a few days, Config::Model now features new cme() and modify() functions that have a behavior similar to cme modify command.

For instance, the following program is enough to update popcon’s configuration file:

use strict; # let's not forget best practices ;-)
use warnings;
use Config::Model qw(cme); # cme function must be imported

The object returned by cme() is a Config;:Model::Instance. All its methods are available for a finer control. For instance:

my $instance = cme('popcon');

When run as root, the script above shows:

Changes applied to popcon configuration:
- PARTICIPATE: 'no' -> 'yes'

If need be, you can also retrieve the root node of the configuration tree to use Config;:Model::Node methods:

my $root_node = cme('popcon')->config_root;
say "is popcon active ? ",$root_node->fetch_element_value('PARTICIPATE');

In summary, using cme in a Perl program is now as easy as using cme from a shell script.

To provide feedback, comments, ideas, patches or to report problems, please follow the instructions from CONTRIBUTING page on github.

All the best

Automount usb devices with systemd


Ever since udisk-glue was obsoleted with udisk (the first generation), I’ve been struggling to find a solution to automatically mount a usb drive when such a device is connected to a kodi based home cinema PC. I wanted to avoid writing dedicated scripts or udev rules. Systemd is quite powerful and I thought that a simple solution should be possible using systemd configuration.

Actually, auto-mount notion covers 2 scenario :

  1. A device is mounted after being plugged in
  2. An already available device is mounted when a process accesses its mount point

The first case is the one that is needed with Kodi. The second may be useful so isΒ  also documented in this post.

For the first case, add a line like the following in /etc/fstab:

/dev/sr0 /mnt/br auto defaults,noatime,auto,nofail 0 2

and reload systemd configuration:

sudo systemctl daemon-reload

The important parameters are “auto” and “nofail”: with “auto”, systemd mounts the filesystem as soon as the device is available. This behavior is different from sysvinit where “auto” is taken into account only when “mount -a” is run by init scripts. “nofail” ensures that boot does not fail when the device is not available.

The second case is handled by a line like the following one (even if the line is split here to improve readability):

/dev/sr0 /mnt/br auto defaults,x-systemd.automount,\
   x-systemd.device-timeout=5,noatime,noauto 0 2

With the line above in /etc/fstab, the file system is mounted when user does (for instance) “ls /mnt/br” (actually, the first “ls” fails and triggers the mount. A second “ls” gives the expected result. There’s probably a way to improve this behavior, but I’ve not found it… this is now fixed).

While testing automount, I realized that the unit may not be started after the daemon-reload command. To check the status of the created unit, run:

sudo systemctl status mnt-br.automount
● mnt-thumb.automount
   Loaded: loaded (/etc/fstab; generated)
   Active: inactive (dead)
    Where: /mnt/br
     Docs: man:fstab(5)

In the example above, the unit was not started. Instead of rebooting, you can start the unit:

sudo systemctl start mnt-br.automount
● mnt-thumb.automount
   Loaded: loaded (/etc/fstab; generated)
   Active: active (waiting) since Sat 2018-03-31 12:09:53 CEST; 2s ago
    Where: /mnt/br
     Docs: man:fstab(5)

“x-systemd.*” parameters are documented in systemd.mount(5).

Last but not least, using a plain device file (like /dev/sr0) works fine to automount optical devices. But it is difficult to predict the name of a device file created for a usb drive, so a LABEL or a UUID should be used in /etc/fstab instead of a plain device file. I.e. something like:

LABEL=my_usb_drive /mnt/my-drive auto defaults,auto,nofail 0 2

All the best