New developments at Opensource.com admin Tue, 06/06/2023 - 13:30

4 Comments

These comments are closed.

Tips for running virtual, in-person, and hybrid events rpaik Wed, 05/03/2023 - 03:00

Comments are closed.

These comments are closed.

Generate web pages from Markdown with Docsify-This paulhibbitts Tue, 05/02/2023 - 03:00

Comments are closed.

These comments are closed.

How I used guilt as a motivator for good its-surya Fri, 04/28/2023 - 03:00

Comments are closed.

These comments are closed.

3 reasons to host a docathon for your open source project lmalivert Fri, 04/28/2023 - 03:00

Comments are closed.

These comments are closed.

Run a virtual conference using only open source tools mairin Thu, 04/27/2023 - 03:00

Comments are closed.

These comments are closed.

3 key open source challenges in developing countries Ahmed Sobeh Thu, 04/27/2023 - 03:00

Comments are closed.

These comments are closed.

Test your Drupal website with Cypress cobadger Wed, 04/26/2023 - 03:00

Comments are closed.

These comments are closed.

5 open ways to help UX designers and developers collaborate better kriker Wed, 04/26/2023 - 03:00

Comments are closed.

These comments are closed.

What's new in GNOME 44? Jim Hall Tue, 04/25/2023 - 03:00

Comments are closed.

These comments are closed.

Darktable 3:RGB or Lab? Which Modules? Help!

Original post in French by Aurélien PIERRE, edited by the pixls community.

Darktable is slowly converging to a scene-referred RGB workflow. Why is that? What does it involve? How does the use of darktable change? Answers here…

This article begins with a 3 section introduction of the Lab space. You don’t need to understand it in detail in order to understand what happens next.

What is Lab?

The color space CIE Lab was published in 1976 by the International Commission on Illumination (CIE), in an attempt to mathematically describe the color perception of the average human being. Lab space aims to decouple the brightness information (L channel) from the chroma information (channels a and b) and takes into account the non-linear corrections that the human brain makes to the linear signal it receives from the retina. Lab space is derived from CIE XYZ space, which represents the physiological response of 3 of the 4 types of photo-sensitive cells in the retina (the cones).

The XYZ space represents what happens in the retina, and Lab represents what subsequently happens in the brain, but both color spaces are models, that is, attempts to describe reality and not the reality itself. There are always discrepancies between a model and reality, but these models are refined and improved as research progresses. Moreover, a model often represents reality only under certain conditions and assumptions, which define the area of validity of each model.

Regarding their respective areas of validity, XYZ works well almost all the time, Lab only works as long as the image has a contrast less than 100:1 (i.e. a maximum dynamic range of 6.5 EV). In the context of the creation of the Lab model in 1976, researchers were working with scanned negatives, and color negatives have a dynamic range of 6 to 7 EV. 6.5 EV is also the static contrast of the retina, and it was a little after 1976 that we realized that the brain was constantly performing HDR fusion of several images per second, meaning that static contrast as a model parameter doesn’t make much sense in the context of human vision.

What is CIE Lab for? It is intended to predict the perceptual difference between 2 colors (the delta E) and to make gamut adaptations when converting an image from one color space to another. One can then try to remap the gamut to the closest color in the target color space via strategies that minimize the delta E digitally.

The big disadvantages of Lab are:

1. It doesn’t work well for strong contrast (> 7 EV), and especially outside the range [1:100] Cd/m²,
2. It is not linear in hue, i.e. if one fixes a pixel’s a and b chromaticity components and changes only its brightness L, the same hue would be expected at a different brightness (this was the design purpose of the Lab space), however there is a slight shift in the hue, more or less marked depending on the original color of the pixel.

What is Lab doing in darktable?

The original idea was to allow separate manipulation of the brightness and chromaticity. In 2009, the year of the project’s creation, cameras had dynamic ranges quite close to Lab’s valid range; the idea was far from bad at the time, especially because darktable did not have a complex masking option then.

Advantages:

1. Lab, being a reference space and therefore independent of the display color, makes presets very easy to set up and transfer,
2. Lab sets the middle gray (18%) to 50%, so the interface is more intuitive (the middle gray is in the middle of the graph of the tones, for example).

Problems:

1. Today’s cameras have dynamic ranges that are largely outside of the conditions under which Lab is valid, which makes the defects of this space more apparent. With dynamic ranges from 10 to 14 EV at 100 ISO, any recent camera does HDR by default, and Lab is not designed to handle that much dynamic range
2. Pushing pixels in Lab space is very risky, especially when tackling compositing and image fusion with softened and feathered masks. We’ll get back to that, but it has to do with the next problem…
3. Lab is not adapted to physically realistic corrections, such as blurring, deblurring, denoising, and any filter that simulates or corrects for an optical effect.

In brief, Lab was a youthful mistake. That said, all other photo processing pieces of software seem to work by default in non-linear RGB spaces (with a “gamma” applied at the beginning of the pipe) that are basically equivalent (regarding their flaws and drawbacks for image filters).

How does Lab work?

Everything (e.g. the camera sensor) starts from a linear RGB space. We convert linear RGB to XYZ. For the purposes of the demonstration, we can consider the XYZ space as a special RGB space whose primary colors have been slightly manipulated (that’s not the case, but it behaves the same way). XYZ is also a linear space.

We then switch from XYZ to Lab by applying a “gamma correction” on the luminance channel (from Y to L), and a rotation on the channels a and b. Mathematically, Lab is like applying 2.44 gamma to linear RGB – it poses the same practical problem: it’s highly non-linear.

Summary

Lab doesn’t work for high-contrast images and doesn’t work well for images with moderate contrast. It encodes pixel values in a perceptual manner rather than physical one, which will pose a problem in the following. Lab was not designed for image processing, but only as a way to study human vision.

Precision: I have used the term “gamma” or “gamma correction” incorrectly here. Strictly, a gamma function is the specific (technical) electrico-optical transfer function (EOTF) of old-school CRT screens, which is a power function with an exponent between 1.8 and 2.2. Nowadays, people incorrectly name “gamma” any power function used for technical integers encoding or artistic lightness adjustments, which is confusing. Any encoding transfer function (using a power function or not) should be called OETF (Opto Electrical Transfer Function), and is used only to alleviate the limits of 8 bits integer file formats. Any artistic power-like brightness corrections should be called a tone curve. Even if the operation is the same, it does not have the same meaning and should not be applied at the same place in the graphics pipe. But ICC nomenclature continues to call “gamma” the exponent used to encode/decode RGB pixels when using integer file formats, so here we are, mixing unrelated concepts under an umbrella name just because the maths write the same. But, when communicating with people out of the industry, it’s often easier to use the incorrect name so that everyone sort-of understands, even if it carries on the confusion.
By the way, power-like OETF are completely unnecessary as long as you use floating point arithmetic and files format (32 bits TIFF, PFM, OpenEXR…).

The limits of non-linear spaces in image processing

First of all, what do we mean by “linear”? If y is linear with respect to x, it means there’s a relationship between x and y in the form y= a . x + b, where a and b are real constants. Linear means proportional to something plus or minus a constant.

So, when we talk about linear RGB space, we mean that the RGB values are proportional to something. But proportional to what?

The sensor counts the number of photons it receives at each photosite. Every pixel contains information on the light spectrum captured at its position, in the form of 3 intensities (red, green, blue). The coefficient of proportionality a between the number of photons and the final RGB value is the ISO sensitivity of the sensor. The constant b is the sensor noise threshold. The RGB signal is proportional to the energy of the light emission picked up by the camera sensor.

From the point of view of human perception, these intensities being proportional to the physical energy level of the light emission, does not make sense. In fact, the brain applies a non-linear, logarithmic correction that the Lab color space approximates using a cubic root. This means that we have an increased sensitivity to dim light, and reduced sensitivity to bright light.

However, all optical operations that are performed during image capture (e.g. lens blur, noise creation, or the effect of a color filter added to the lens) are applied directly to the photons. To reverse the lens blur or to simulate it when processing, we need to work on the linear RGB information, which is the closest thing to the photon data that is available to us.

See for yourself: Which one of these two computer generated bokeh (original below) seems the most natural to you? (See also a more spectacular example on Chris Brejon’s website)

Observe in particular how the dark silhouettes (bottom left) merge into the light background, or the contrast of the pentagons formed by the lens diaphragm on spotlights.

Another example, with a simple blur on smooth surfaces: Which of these gradations seems to you to be the most progressive?

These two examples were generated with Krita, which allows you to work in both linear and non-linear RGB, and has filter layers including a physically realistic lens blur.

This type of problem will occur the same way in darktable, as soon as you use the modules sharpen, high-pass, low-pass, and feathering/smoothing of drawn and/or parametric masks (which are blurs).

Blurring, deblurring, or anything else connected to optics must take place in linear RGB. There’s no mathematical model* that allows correct gradients in RGB encoded for display (with an OETF) or in Lab, due to loss of connection between pixel values and light energy.

* and just because the problems aren’t visible all the time doesn’t mean the problems aren’t always there. We can, up to a certain point, hide them with mathematical trickery (thresholds, opacity, etc.), but they will always end up coming out at the worst time. Trust me, I know exactly where to push to make it break.

This is also the problem that arises with hue zones blending in the color zones module (even if a tweak, introduced under the “smooth” process mode, attempts to hide this under the rug), which produces granular and sharp transitions.

The only darktable module that works in Lab to make a blur, and where it still works reasonably, is the local laplacian mode of the local contrast module. The price we pay for it to work is that it’s very computationally heavy and theory is like rocket science. And, even if the blur is stable, it comes with an ungracious desaturation with a hue shift to muddy grey-blue when you push the sliders a little too hard.

The benefits of a linear RGB treatment

So here’s where you say “as long as I’m not blurring my images or working only on color, I can still use Lab”.

That’s partly true, but in fact, even in those cases, working in linear RGB is simpler, with faster algorithms that can tolerate more extreme adjustments without showing annoying side-effects. Also, once again, Lab can’t support high dynamic ranges, so care must be taken to use the Lab modules after HDR tone mapping.

Strictly speaking, the only application where Lab is required is the gamut mapping, when changing color space before sending the image to a file or to the screen. And even then, since 1976, better spaces have been developed (IPT-HDR, JzAzBz) for this purpose, in HDR and with an almost perfect linearity of hues.

The current state of darktable

With the release of darktable 3.0, the default pipeline (i.e. the basic module order) has been reordered around filmic RGB. There are 4 essential steps in this pipe:

1. the demosaic module, which converts the raw file (which only contains the intensity of a single layer, R, G or B at each pixel site) to a picture (with complete RGB data for each pixel location),
2. the input color profile module, which converts the sensor’s RGB space to a standard working color space,
3. the filmic RGB (or the base curve) module, which translates between linear space (proportional to light energy) into non-linear (perceptually compressed) space,
4. the output color profile module, which converts from the standard working space to the RGB space of the screen or the image file.

Note that the base curve approach remains the one applied by default because it allows darktable to more-or-less approximate the rendering of the camera JPEG as soon as the software is opened, which seems to be the preference of many users. Nevertheless, as part of darktable 3.0, the base curve was pushed back in the pixel pipe by default, to just before the filmic RGB module, which makes it safe for the colors produced by the modules that are applied earlier. The base curve module also was provided with a color preservation mode, which produces results similar to filmic RGB. Between base curve and filmic RGB, for darktable 3.0, the difference is now only about ergonomics and on the ability to recover very low light. filmic RGB is a little more complex to understand but faster to set up (once properly understood), and is more powerful when working in deep shadows.

Modules that work in linear RGB and output in linear (thus leaving the pipeline linear after them) are:

1. exposure
2. white balance
3. channel mixer
4. tone equalizer (which is linear in parts).

The advantage of performing linear operations is that they do not affect the chrominance of the image (because changing the luminosity leaves the chrominance intact) and preserve the energy proportionality of the signal. These modules must be positioned before filmic RGB or the base curve. Exposure and tone equalizer are recommended prior to the input color profile. They can be used safely and without moderation. Note that there is a catch here on the tone equalizer, which preserves local linearity (within the image areas), but not the overall linearity (between zones). It corresponds to what would happen if we walked onto the scene with a flashlight, and hand reilluminated the objects in the scene, so we still keep the physical coherence of the signal.

Modules that work in linear RGB and carry out non-linear, but chrominance-preserving operations, (provided that the chroma preservation mode is activated) are:

1. RGB curves
2. RGB levels

The chrominance is preserved via methods that constrain the RGB ratios in and out of the module, so as to keep them identical. Note that RGB curves and RGB levels can be moved before or after filmic RGB depending on the intention, since they’re doing non-linear operations anyway. On the other hand, be careful not to use the mask feathering on modules that come later, as linearity is no longer assured and mask blurring + blending could produce unpleasant results.

Modules that work in linear RGB and carry out non-linear operations without preserving the chrominance are:

1. local tone mapping (we’ll get back to that)
2. color balance
3. LUT 3D

Color balance is designed to be applied to linear RGB data that hasn’t been corrected for contrast, i.e. before filmic RGB, tone curves etc. It does not preserve the chrominance because its explicit purpose is to adjust chrominance creatively. Similarly for LUT 3D, for which the main goal is to emulate analog film emulsions or complex aesthetic transforms.

I remind readers here that filmic RGB is a dynamic range compressor, from the high dynamic range of the camera to the low dynamic range of the screen. It is not a tone curve intended to apply an artistic correction, but a mapping of tones to force fit the sensor data into the the available screen space. filmic RGB tries to protect the details as much as possible (which we assume a priori are in the middle tones) and to keep a certain optical readability in the image.

Before filmic RGB, in the linear pipe, we still find some modules that work in Lab but perform linear operations that should (strictly speaking) be realized in linear RGB:

1. contrast equalizer
2. high pass
3. low pass
4. sharpen
5. denoise (non-local means)

These modules need to be adapted in the future to be able to work on a linear Yxy space (derived from CIE XYZ) because it is a mistake to make them work in Lab (at least, as a default). It’s a relatively easy job to do, because Yxy breaks down the luminance (Y channel) and chrominance (channels x and y) with a logic similar to Lab, minus the non-linear transformation. In the meantime, you can continue to use them, but with moderation. For the contrast equalizer, note that it uses an edge-sensitive wavelet separation, which makes it quite cumbersome to execute, but very effective at preventing halos, even considering that it works in Lab.

After filmic RGB, in the non-linear pipe, are all the other Lab modules, since they require low dynamic range. Some of these modules could also be converted to xyY and moved before filmic RGB in the future (in particular the soften, grain and fill light modules). Also note that the vignette module was left at the end of the pipe, as before, even though it works in RGB. It’s likely it’ll be better off before filmic RGB, or even before the input profile, but its code is surprisingly complex for what it does, and I haven’t had the time to unravel the imbroglio in order to understand what its working hypotheses are.

Modules not recommended

A number of modules are not recommended due to fundamental errors in design (based on my personal opinion, which is based on my practical and theoretical experience in image retouching), and in the spirit of streamlining the workflow with a minimum number of steps. There is nothing stopping you from continuing to use them, especially since users regularly introduce me to new use cases that I hadn’t thought of. But the idea here is to give you the keys to the best possible result as quickly as possible with as little fuss as possible.

Local Tone Mapping

Local tone mapping internally encodes RGB values logarithmically (they are then decoded at the output, so no problem at at this level), then applies a bilateral blur to these logarithmic values. As we saw above, theory is clear: a blur, on anything non-linear, produces halos and fringes. And as promised, the default setting range of this module is much reduced, so that users have become accustomed to merging the output of the module with low opacity – this is only hiding the misery.

Prefer the tone equalizer.

Global Tone Mapping

This module works in Lab color space to perform HDR compression, and if you have followed my explanations, you will understand that this is a contradiction in terms. In addition – and this is important – the white value is adjusted automatically from the maximum in the image, so the overall brightness of the image may change depending on the size of the export, due to the smoothing effect of the setting to scale (interpolation). To be expected: a lighter or darker JPEG than the preview in the darkroom.

Prefer filmic RGB.

Shadows and highlights

Similarly, this module works in Lab color space to perform HDR compression and uses a Gaussian or bilateral blur to isolate highlights and shadows. In practice, it gives halos quickly as soon as you push the parameters (even if the bilateral blurring lessens the problems a little), and it even tends to add local contrast (as a secondary effect) in the highlights, giving clouds a very HDR look. In the shadows, used a little hard, colors turn blue-grey. In practice, it does not work, except for minor corrections.

Prefer the tone equalizer.

Low-pass filter

The low-pass filter is actually a simple blur. A lot of people use it to invert the contrast, and then blend it with overlay or soft/hard/linear light, to compress the dynamic range. This is in fact exactly what the shadows and highlights module already does in fewer steps for the user. As mentioned above, the low-pass module works in Lab color space, so for the blur… Expect the worst.

Prefer the contrast equalizer for blur, or the tone equalizer for local dynamic range compression

High-pass filter

A lot of people use the high-pass module by blending it with overlay or soft/hard/linear light, for adding sharpness. This is in fact exactly what the sharpen module already does. The high pass is achieved by subtracting between a blur (low-pass) and the original image, so we have the same problem as for the low-pass because it’s still working in Lab.

Prefer the contrast equalizer for fine sharpness, or the local contrast for the general sharpness.

Sharpen

The sharpen module was originally intended for sensors with an optical low-pass filter as well as the smoothing due to demosaicing in some cases. First, as this module works in Lab, you need to avoid pushing it so much that it produces halos. Second, the internal sharpening method (using unsharp mask) is rather archaic and quickly artificial, even in RGB mode. Thirdly, in view of the sharpness of modern optics, given that many sensors no longer have low-pass filters, and that most of the photos will be exported at a reduction ratio of at least 8:1 (24 Mpx sensors to 3 Mpx screen), pixel-level sharpness enhancement has become practically useless. Generally speaking, the digital photographer of the 21st century would benefit from calming down with the crisp sharpness - it would be good for everyone.

Prefer to the contrast equalizer to deflect the optics via the presets provided, or the local contrast for general sharpness.

Monochrome

The monochrome module works in Lab, which it uses to define a weighted contribution of certain colours to the density of the black, in order to convert color into shades of gray. The problem is that the interface is quite sensitive to the settings, and a small correction can produce large changes and break the overall contrast in a rather ungraceful way. In practice, getting a predictable result is quite difficult and this module often results in a lot of tedious micro-adjustment sessions.

The idea of a weighted contribution of colors to the density of black comes from silver film, which behaves exactly the same way as this. But, as you saw coming, film doesn’t work in Lab and is not perceptually realistic. This idea is taken up in a physically realistic way in the channel mixer module, where several emulsion presets of commercial silver film are offered to create a grey channel. Note that, in order for the coefficients to be accurate, the colour space of the operating mode (in the module input profile) must be set to REC 709 linear, otherwise the settings will have to be adjusted.

For a black and white treatment that is based on human perceptual luminance (linear), simply lower the input or output saturation to 0% in the color balance module (right-click on the slider and enter 0 on the keypad – the setting is only up to 50% by default in the interface).

Prefer the channel mixer for a silver approach or the color balance for a perceptual approach.

Fill light/Bloom/Zone System

These three modules aim to re-illuminate a part of the image, and attempt to dilute the correction in intensity and in space by blurring in the picture. But since they’re working in the Lab color space …I won’t say it again… The results are just bad all the time, except with very soft settings, in which case you didn’t’ really need those modules in the first place.

Prefer the exposure module with masks, or the tone equalizer

Color Correction

Every photograph has at least two sources of light: a direct source (lamp, sun, candle) and a reflected source (walls, clouds, floors, ceiling). It often happens that the white balance of these two sources does not coincide. In practice, human vision has ways to correct for this this, but not the camera. So it requires a separate white balance correction for the highlights (which generally receive direct light) and the shadows (which usually receive reflected light).

This is what the color correction module offers you, again in Lab color space, and with mixed and unnatural results as soon as you push the adjustment. When you think about it carefully, the white balance can be reduced to discussions of light spectrum, and the correction is simpler in RGB, especially to manage progressively of correction.

The color balance module allows you to adjust this quickly, and not just for the shadows and the highlights, but also for midtones. Using the color-pickers, to the right of the tint sliders, it also allows you to go directly to sample neutral tones in the image (for black, gray and white) and let the software calculate the complementary color. See the manual for more details.

Prefer color balance.

Velvia

Velvia works in RGB and works on a logic quite similar to the color balance saturation. On the surface, it smells good. Except that in fact, its colorimetric equation is not perceptually correct. What it’s doing is changing the saturation (which is its intention), but at the same time it also changes the hue and brightness (which becomes awkward). The problem is that it seems to have been optimized for non-linear RGB. As a result, it is the kind of module that is typically unpredictable.

Prefer color balance.

Levels/RGB Levels

These two are working as they should, no problem with that. But when you look at the code, you can see that it duplicates exactly the slope/offset/power mode of the color balance module. The white point is scaled by a simple exposure correction, such as the slope factor or even the exposure of the exposure module. The black point is adjusted by adding a constant, such as the factor of the offset, or the black level correction of the exposure module. The grey point is adjusted by a power function (sometimes improperly called gamma), just like the power factor of the color balance. They are not just the same features, they are exactly the same math. The difference is therefore not only in ergonomics, but also in the fact that the color balance gives you the numerical value settings, making them more easily transferable from one image to another or from one application to another. Curves and levels also assume you work SDR images, with data encoded between 0 and 1. If you work HDR pictures or raised the exposure quite a lot earlier in the pipe, the pixel values will not be clipped, but the GUI will not give you control over the pixels above 1 (or 100 %).

If you already use the color balance, there is no need to add an additional level module. Finish your retouching in the same module.

Curves/RGB Curves

These also work well, but considering their classic use … are they really useful? Usually they are used to add/remove brightness, which falls in the same use case as the grey of the levels module or the power of the color balance module, or to add/remove contrast, which can be adjusted or by decreasing/increasing the interval between white and black (in a linear way) or by applying a non-linear brightness compression, again available from color balance.

Curve ergonomics is a real problem in an RGB linear workflow, because the middle gray is assumed to be in the center of the graph, which therefore assumes that we are working in non-linear RGB (where the gray at has been increased to 50%). In a linear encoding, the standard medium grey is expected at 18% (but the practice depends or where you anchored your exposure in camera), and the contrast control around this value not being centered on the graph becomes complex in the interface. In addition, the graph of the curves assumes a limited RGB signal between the values 0 and 100% (or 1)… 100% of What? White screen luminance. In a linear workflow, the HDR signal can go from 0 to infinity, and it is at filmic RGB step that we’re in charge of putting everything back between 0 and 100% of the white screen.

The contrast in the color balance module is compatible with this approach using the contrast fulcrum parameter, which allows the selection of the contrast reference. Thus when changing the contrast, we increase the light above the fulcrum, and reduce it below, but the fulcrum remains unchanged. The display workflow (in Lab or non-linear RGB) always has the implicit assumption that gray is 50%, uses it as a contrast reference, and doesn’t allow you to change that value.

Prefer color balance.

Contrast/Brightness/Saturation

Module working in Lab, which duplicates again the modules levels, curves, and color balance while adding undesirable effects on colours.

Prefer color balance.

Modules to be used with care

There is no correct replacement for the following modules for the moment, but they should be used with caution because they can be unpredictable and can cause you to lose a lot of time.

Vibrance

Vibrance works in Lab by applying a saturation correction that penalizes already saturated pixels to avoid over-saturation, but also tends to darken colors. The result is far from ugly, but the problem is that we can’t control how much we darken for the amount we resaturate.

Prefer color zones with a selection by saturation.

Color zones

This module would be awesome if the merging of colour zones were more progressive. It now has two processing modes (strong, the old, and smooth, the new) which are trying to meet this challenge of two different ways, resulting in transitions too discrete for the new, and too abrupt for the old. One more time, it works in Lab, when similar functionality in Capture One seems to be using HSL or HSV, which seems to perform better than Lab.

In some cases, color zones will benefit from being replaced by the color balance module where parametric masking may be used to isolate the shades you want to act on. Then the refinement of the guided filter parametric mask should help in difficult cases. For the rest, color balance allows us to change the shade, saturation and brightness exactly the same.

Note, however, that the color balance module, although working in RGB internally, merges the masks into Lab because this module is older than the possibility to have 100 % RGB modules, and converts from Lab to RGB internally. We’re still working on it…

Prefer color balance.

Vignetting

Adding a vignette around an image is not complicated: you just have to to gradually lower the exposure, and eventually the saturation with a drawn mask. However, the vignetting module performs incomprehensible black magic, which is much more complicated than that, with an internal homogenization which would be superfluous if things were well done. The result is rarely natural, the transition in luminosity being too violent compared to a real vignette.

You will get better results with an instance of the exposure module set to -0.5 EV, a circular mask with a large transition area whose polarity is reversed, possibly coupled with a desaturation in color balance to which you pass the same mask as used in exposure (via a rasterized mask).

Prefer the exposure (and, optionally, the color balance saturation) modules.

Mask blend modes not recommended.

Few people know this, but the blend modes lighten, darken, overlay, soft light, hard light, pin light and linear light implicitly expect the grey level to be 50% grey and are thus totally connected to the display-referred workflow. The blend modes are going to treat the pixels differently depending on whether they are above or below 50 %. Remember that the linear RGB workflow keeps the gray point at 18% (or even less). These blend modes will therefore behave in a way that is unpredictable in the scene-linear portions of the pipe.

In linear RGB, you should only use blend modes based on arithmetic operations (addition, multiplication, division, subtraction, average), on maximum/minimum comparisons (screen) or on channel separations (hue, color, chroma, etc.).

Note that the multiply mode is one of the most powerful in linear RGB. For example, to enhance the contrast of an image in a natural way, it is enough to use an instance of the exposure module blended with multiply. Set the exposure between 2 and 3 EV and the opacity between 10% and 50%. Exposure is then used to control the pivot of contrast, and opacity the intensity of the effect. It’s fast, simple and effective.

A minimal workflow for beginners

In darktable, you can choose between many modules that allow you to do the same thing in a lot of different ways. But this is merely an illusion of choice, as many of them have more disadvantages than advantages (provided you want to achieve predictable results for demanding edits). If you open the code for any of the modules not recommended above, you will see that they are almost all dated 2010-2011 - the only reason we retained them was to maintain compatibility with edits performed in prior versions of darktable.

You can perform at least 80% of your processing with just 4 modules :

1. exposure
2. white balance
3. color balance
4. filmic RGB

The reason they’re so powerful is because they’re actually extremely simple, when you look at their equations:

• Exposure: RGB_output = exposure × RGB_input + black level
• Color balance :
  • Slope/Offset/Power: RGB_output = (slope × RGB_input + offset)^power
  • Contrast: RGB_output = (RGB_input / pivot)^(contrast × pivot)
• White balance: RGB_out = coefficients × RGB_in
• filmic RGB is a little more complex, but it’s still high-school level math

With these 4 modules, you have everything you need to produce a correct image in terms of colorimetry, contrast, and artistic intent. Remember to turn off the base curve if you use the filmic RGB module. Then, if needed, finalize your edit with the following modules:

• To improve sharpness, the best option is the local contrast module in local laplacian mode
• To deblur the lens, you have deblur presets, more or less pronounced in the contrast equalizer
• To denoise, the best algorithm is in the denoise (profiled) module. Use non-local means auto mode if you don’t want to break your head
• To remove haze, you have haze removal
• To convert to black and white, the easiest way is to use the film presets in the channel mixer
• For creative control of overall contrast and re-lighting of the scene a posteriori, use the tone equalizer module

Some of the following modules have an underestimated power, and they are vastly underutilized:

1. The exposure module, with its masks, can replace all the the other methods of mapping HDR, shadows and highlights the tone equalizer, and even the tone curve and the local contrast (to some extent, when used with blend mode multiply)
2. The channel mixer module can overcome all your gamut problems, including problems with blue in stage lighting, without having to use a fake input profile, but also turn grass into snow or summer trees into fall trees
3. The color balance module can allow you to emulate the colors of a film, compensate for uneven white balance, remove redness on the skin, accentuate the depth and shape, create a split-toning effect, or to give an apocalyptic atmosphere to your images

Finally, to display only a minimal selection in the interface and modules, to the right of “More modules“, open the list of presets and select “workspace: all-purpose“.

darktable is a lot simpler when you understand that you don’t have to use all 77 of its modules at once …

If you have any doubts about the order of the modules, you should know that the default order for version 3.0 has been considered globally, and, apart from some uncertainties on the best position of the vignetting and monochrome modules, the rest is pretty solid, in theory and practice.

Conclusion

Pushing pixel values in either direction is one thing. Merging the corrections so they blend seamlessly together on the whole is another. We’ve seen that Lab or non-linear RGB allow the pixels to be pushed more or less correctly, but that it is always when doing mask blending (aka occlusion) and feathering (aka blurs) that we’re paying the price. It turns out there are a lot of blurs under the hood of darktable, sometimes where you don’t expect them. It’s especially problematic when you’re compositing, e.g. inlaying one image within another, to exchange their background without touching the foreground. And it’s precisely this kind of manipulation that led the movie industry to migrate to a scene-referred linear workflow about twenty years ago.

So darktable is in transition. It’s long, it’s sometimes painful, there are a lot of little bits to change in different places along with grumbling users who are hungry for consistency. At least now you know the why and the how. You also know what you have to win. I hope this helps you move forward.

For new users, limit yourself to the above recommended modules, and venture further when you begin to be comfortable. For older users, the new modules have a lot to offer to you, but old Lab modules are still relevant for moderate creative effects and when used with knowledge of their dangers.

The linear toolbox is being expanded. On the agenda:

• rewriting the 100% RGB color balance (including the blending), with the addition of vibrance (and a vibrance equation home-developed to preserve the color)
• conversion of the contrast equalizer and soften modules to the linear xyY space (because in fact, the Orton effect, on which the soften module is based, is very useful when it works correctly)
• a color equalizer, similar to the tone equalizer, which will allow you to adjust saturation, vibrance and Abney effect according to the pixel luminance, to pep up the filmic RGB curve
• a brand-new lens deconvolution module, respectful of the depth of field (but for that, I need to develop a special wavelet based on the guided filter), which should turn your soft 18-55 mm into a Zeiss for much less
• and of course the OpenCL version of the tone equalizer

There is more work than people to do it, so wish us good luck, don’t forget to support us, and Happy New Year 2020 to all of you!

Portrait photographer in Nancy-Metz. Calculation specialist, modeling and numerical simulation for image processing (denoising, deblurring, colour management) and thermal engineering. Developer of filmic RGB, tone equalizer, color balance, and the new themeable interface for darktable 3.0. darktable user since 2010. darktable is my job, so help me out to develop.

G'MIC 2.7 - Process Your Images with Style!

The IMAGE team at the GREYC research laboratory is pleased to announce the release of version 2.7 of G’MIC (GREYC’s Magic for Image Computing), its free, generic, extensible, and probably a little magical, framework for digital image processing.

The previous PIXLS.US article on this open-source framework was published a year ago, in August 2018. This new release is therefore a good opportunity to summarize the main features and milestones of the project’s life over the past twelve months. Fasten your seat belts, the road is long and full of surprises!

---

Useful links:

• The G’MIC Project
• G’MIC Twitter Feed
• G’MIC Forum on PIXLS.US

---

1. G’MIC in 300 words

G’MIC is a piece of software that has been developed for more than 10 years now, mainly in C++, by two members of the IMAGE team of the GREYC lab: Sébastien Fourey and David Tschumperlé. It is distributed under the terms of the CeCILL free-software license. GREYC is a French public research laboratory located in Caen, specialized in digital sciences, under the head of three academic institutions: CNRS, University of Caen, and ENSICAEN.

The IMAGE team, one of the seven teams in the laboratory, is composed of researchers, professors, Ph.D. students and engineers, all specialized in the fields of algorithmics and mathematics of image processing.

G’MIC is cross-platform (GNU/Linux, MacOS, Windows, …). It provides various user interfaces for manipulating generic image data, i.e. 2D or 3D hyperspectral images or sequences of images with floating-point values (which indeed includes “usual” color images). Around a thousand different processing functions are already available. However, arbitrarily many features can be added thanks to an integrated scripting language.

The most commonly used G’MIC interfaces are: the gmic command, that can be accessed from the command line (which is an essential complement to ImageMagick or GraphicsMagick), the G’MIC Online Web service, but above all, the plug-in G’MIC-Qt, available for the well-known image editing software GIMP, Krita, and Paint.net. It provides more than 500 different filters to apply on images.

Thanks to its extensible architecture, G’MIC is regularly enhanced with new image processing algorithms, and it is these latest additions that will be discussed in the following sections.

2. Add style to your images!

G’MIC has recently implemented a neat filter for style transfer between two images, available from the G’MIC-Qt plug-in under the “Artistic / Stylize“ entry. The concept of style transfer is quite simple: we try to transform an image (typically a photograph) by transferring the style of another image to it (for example a painting).

The implementation of such a style transfer method is relatively complex: The algorithm must be able to recompose the original photograph by “borrowing” pixels from the style image and intelligently combining them, like a puzzle to be reconstructed, to best match the content of the data to be reproduced, in terms of contours, colors and textures. How easily this is done depends of course on the compatibility between the input image and the chosen style. In computer graphics, most existing implementations of style transfer methods are based on convolutional neural networks, more particularly generative adversarial networks (GANs).

G’MIC implements style transfer in a different way (without relying on neural networks, the scientific article detailing the algorithm is currently being written!). This method is parallelizable and can therefore benefit from all the processing units (cores) available on the user’s computer. The computation time naturally depends on the input image resolution, and the accuracy of the desired reconstruction. On a standard 4-cores PC, it could take tens of seconds for low resolution images (e. g. 800x800), up to several minutes for larger pictures.

As one can imagine, it is a very versatile filter, since we can apply any style to any input image without hard constraints. Some famous paintings are available by default in the filter, in order to propose predefined styles to the user.

Let us be honest, it is not always easy to obtain satisfactory results from the first draft. It is generally necessary to choose your starting images carefully, and to play with the many parameters available to refine the type of rendering generated by the algorithm. Nevertheless, the filter is sometimes able to generate quite interesting outcomes, such as those shown below (the original photo is visible at the top left, the style chosen at the top right, and the result of the style transfer at the bottom). Imagine how long it would take for a graphic designer to make these transformations “by hand”!

Other examples of image stylization can be found on the image gallery, dedicated to this filter. To our knowledge, G’MIC is the only “mainstream” image processing software currently offering a generic style transfer filter, where any style image can be chosen.

A last funny experiment: get a picture of an Alien’s head, like Roswell, and then select a crop of the Mandelbrot fractal set as your style image. Use the transfer filter to generate a “fractal” rendering of your alien head. Then, make the whole world believe that the Mandelbrot set contains the mathematical proof of the existence of aliens… ☺

In short, this filter has a clear creative potential for all kind of artists!

3. Interactive deformation and morphing

This year, G’MIC got an implementation of the RBFs interpolation method (Radial Basis Functions), which is able to estimate a dense interpolated function in any dimension, from a known set of scattered samples (not necessarily located on a regular grid). Thus, it gave us the idea to add distortion filters where the user interaction is focused in adding and moving keypoints over the image. In a second stage, G’MIC interpolates the data represented by these keypoints in order to perform the distortion on the entire image.

Let us start with the “Deformations / Warp [interactive]” filter which, as its name suggests, allows the user to distort an image locally by creating/moving keypoints.

The animation below shows this interactive filter in use, and illustrates the fact that these keypoints can be considered as anchors to the image, when they are moved.

(For those who might be concerned about the portraits photos used in the figures above and below: all these portraits are totally artificial, randomly generated by GANs via the website This Person Does Not Not Exist. No moral prejudices to dread!).

The great advantage of using RBFs-based interpolation is that we do not have to explicitly manage a spatial structure between the keypoints, for instance by defining a mesh (i.e. a “deformation grid”). This gives a greater degree of freedom in the obtained distortion (see Fig.3.3. below). And at the same time, we keep a rather fine control on the local amplitude of the applied distortion, since adding more “identity” keypoints around a region naturally limits the distortion amplitude inside this region.

A short demonstration of this distortion filter is also visible in this Youtube video.

And why not extending this kind of distortion for two images, instead of a single one? This is precisely what the new filter “Deformations / Morph [interactive]” does. It is able to render a morphing sequence between two images (put on two separate layers), using the same interpolation technique that only asks for the user to set colored keypoints which match on both images.

In the example above, keypoints are placed on characteristic areas of both faces (tip of nose, lips, eyebrows, etc.). In practice, this takes no more than 5 minutes. Thanks to these keypoints, the algorithm is able to estimate a global deformation map from one image to another, and can generate temporally “mixed” frames where the elements of the face remain relatively well aligned during the whole morphing sequence.

By comparison, here is what we would obtain by simply averaging the two input images together, i.e. without correcting the displacement of the facial features between both images. Not a pretty sight indeed!

Thus, the morphing filter is able to quickly generate a set of intermediate frames, ranging from the “Source” to the “Target” faces, a sequence that can then be saved as an animation.

Many other use cases of this morphing filter can be considered. The following example illustrates its application to render an animation from two photographs of the same object (a garden gnome), but shot with different DOFs (Depth of Field).

Command line users will be pleased to know that these two filters can be tested very quickly from a shell, as follows:

$ gmic image.jpg x_warp
$ gmic source.jpg target.jpg x_morph
~

4. Ever more colorimetric transformations

For several years, G’MIC has contained colorimetric transformation filters able to simulate the film development process, or to give particular colorimetric moods to images (sunlight, rain, fog, morning, afternoon, evening, night, etc.). In a previous report, we already mentioned these filters, which are essentially based on the use of 3D CLUTs (Color Lookup Tables) for modeling the color transformation.

A 3D CLUT is technically a three-dimensional array that provides for each possible RGB color, a replacement color to apply to the image.

The main interest of these 3D CLUTs is the great variety of transformations they can represent: They can indeed define RGB-to-RGB functions with almost any kind of variations. The only “constraint” of these methods is that all image pixels having the same color will be transformed into pixels that also have an identical color.

The disadvantage, however, is that these 3D CLUTs are relatively data intensive. When you want to embed several hundred different ones in the same piece of software (which is the case in G’MIC), you quickly find yourself with a large volume of data to install and manage. For instance, our friends at RawTherapee offer on their website an additional pack of 294 CLUTs functions to download. All these CLUTs are stored as .png files in a .zip archive with a total size of 402 MB. Even if downloading and storing a few hundred _MB_ is no longer limiting nowadays, it is still quite large for things as simple as color changing filters.

This year, we have therefore carried out important research and development work at the GREYC lab on this topic. The result: a new lossy compression algorithm (with visually imperceptible compression losses) that can generate binary representations of CLUTs with an average compression rate of more than 99%, relative to the data already loslessy compressed. The general idea is to determine an optimal set of color keypoints from which the CLUT can be reconstructed (decompression), and this, with a minimal reconstruction error.

As a result, this original compression method allowed us to offer no less than 763 CLUTs in G’MIC, all stored in a binary file that weights less than 3 MB !

All these color variation filters have been grouped into two separate entries in the G’MIC-Qt plug-in, namely “Colors / Simulate Film” (for analog film simulations), and “Colors / Color Presets” (for other color transformations). Each of these filters provides sub-categories for a structured access to the hundreds of CLUTs available. To our knowledge, this makes G’MIC one of the image processing software with the most colorimetric transformations, while keeping a reasonable size.

Readers interested in the mathematical details of these CLUT compression/decompression algorithms may refer to the scientific paper we wrote about it, as well as the presentation slides that have been presented at the conferences GRETSI’2019 (French conference, in Lille) and CAIP’2019 (International conference, in Salerno).

To finish with this topic, note that we have made an open-source implementation of our decompression algorithm of CLUTs available online (in C++, with 716 CLUTs already included). Discussions have also been initiated for a potential integration as a Darktable module for managing 3D CLUTs.

5. Create palettes by mixing colors

Let us now talk about the recent “Colors / Colorful Blobs” filter which is directly inspired by the original concept of Playful Palette created by the Adobe Research team in 2017. This filter is intended for illustrators (designers and digital painters). The goal: Create color palettes which contain only a few main colors (the ones you want to use in an illustration), but also a few sets of intermediate shades between these colors, in the form of color gradients. An artist is theoretically able to better preserve the color coherence of its artwork, by picking colors only from this palette.

As shown on the figure above, the filter allows the artist to create and move colored “blobs” that, when merged together, create the desired color gradients. The result of the filter is thus an image that the artist can use afterward as a custom 2D color palette.

From a technical point of view, this filter is based on 2D metaballs to model the color blobs. Up to twelve separate blobs can be added and different color spaces can be chosen for the calculation of the color gradient (sRGB, Linear RGB or Lab). The filter also benefits from the recent development of the G’MIC-Qt plug-in that enhances the user interactivity inside the preview widget (a feature we mentioned in a previous report), as seen in the animation below (see also this longer video).

This filter may not be useful for most G’MIC users. But you have to admit, it’s pretty fun, isn’t it?

6. Some more filters

Let us now describe a selection of a few other filters and effects added during the year, perhaps less original than the previous ones (but not completely useless anyway!).

• First of all, the “Rendering / Symmetric 2D Shape” filter is a great help when you want to draw geometric shapes having angular symmetries.

  

  The plane can be subdivided into up to 32 angular pieces, each of which can contain a maximum of six keypoints to define a shape profile, allowing potentially complex and varied shapes to be rendered (such as the super-shuriken below!).

  

• The “Degradations / Self Glitching” filter combines an image with a shifted version of itself, to create a Glitch-art type image. Several bitwise operations (Add, Mul, And, _Or_, Xor,…) can be chosen and you can adjust the shift direction and amplitude, as well as various other controls.

  

  Again, this is not a filter that will necessarily be used every day! But it may be helpful for some people. It was actually added in response to a user request.

• In the same style, the “Degradations / Mess With Bits” filter applies some arithmetic operations to the pixel values, seen as binary numbers (for instance, bit shift and bit inversion). Always with the idea of rendering Glitch art, of course!

  

• The “Degradations / Noise [Perlin]” filter implements the generation of the Perlin noise, a very classical noise model in image synthesis, used for the generation of elevation maps for virtual terrains. Here we propose a multi-scale version of the original algorithm, with up to four simultaneous variation scales.

  

• The “Frames / Frame [Mirror]” filter is also a “tailor-made” effect, to meet the needs of a G’MIC-Qt plug-in user. This photographer wanted to resize his photos to obtain a precise width/height ratio, but without having to crop his images. The solution was instead to add image information at the edges of the picture, by symmetry, in order to obtain the desired ratio. So that’s what this filter does.

  

• Finally, let us mention the upcoming advanced image noise reduction filter, by Iain Fergusson, whose development is still in progress. Iain has been contributing to G’MIC for several years now by implementing and experimenting original denoising filters, and his latest project seems really interesting, with promising results. This video shows this filter in action, a good place to learn a little more about how it works.

Now that we’ve looked at these new filters, it seems important for us to remind that, as in many IT projects, this visible part of the iceberg hides a set of lower-level developments done to improve the interactive possibilities of the G’MIC-Qt plug-in, as well as the performance of the internal scripting language interpreter (the G’MIC language), which is how all these filters and effects are actually implemented. These improvements and incremental slight optimizations of the code base benefit to all filters (even those already available for several years) and it actually represents most of the development time we spend on G’MIC. So, dear users, do not be surprised if no new filters appear for a while. It is probably just because we are doing serious work on the G’MIC framework core!

7. Other notable points in the project life

Here are listed some other important news that have punctuated the life of the project since August 2018.

7.1. We now accept donations!

This is essential news for us: since March 2019, the G’MIC project has been granted permission to collect donations (via Paypal), to help in its maintenance and development!

This is a good thing, because until now, there was no simple way for a public research laboratory as the GREYC, to accept donations for supporting the development of a free software application such as G’MIC, an application used daily by several thousand people around the world. And we have currently no other ways to finance this piece of software in the long term.

Thus, we have partnered with LILA (Libre comme l’Art), a French non-profit organization promoting Arts, Artists and Free Software, who accepted to collect donations for us.

In practice, this is something that has been a little long to set up, but now that the donation system is operational, we hope to benefit from it in the future to make the project development even faster (the possible use of the raised funds is detailed on the donations page, this being of course very dependent on the amount of money collected).

For the sake of transparency, we will post the monthly amount of collected donations on the project website. At this point, we don’t really know what to expect in practice. We will see how these donations evolve. Of course, we would like to thank all those who have already participated (or plan to do so) in supporting our open-source framework for image processing. Our ultimate dream would be, one day, to say that the illustration below is only a distant memory!

7.2. Integrating “Smart Coloring” into GIMP

Let us also mention the work of Jehan, known to PIXLS.US readers as a regular GIMP developer. Jehan has been hired by the GREYC laboratory in September 2018, to work on G’MIC (for a 12-month fixed-term contract), thanks to a grant funded by the INS2I Institute of the CNRS (for which we are grateful).

One of its first missions was to re-implement the G’MIC “Smart Coloring” algorithm (that we had already talked about previously) as a new interactive mode integrated into the existing GIMP “Bucket Fill“ tool.

Jehan described all his work in a blog post, which is strongly recommended for reading. Of course, we don’t want to copy his post here, but we want to mention this activity, and to consider it as another original contribution of the G’MIC project to free software for graphic creation: at the GREYC laboratory, we are really happy and proud to have imagined and developed an image colorization algorithm, which artists can use through a well integrated tool into such a popular piece of software as GIMP!

This intelligent colorization algorithm has been the subject of scientific publications, presentations at the conferences GRETSI’2017, EuroGraphics VMV’2018, as well as at the Libre Graphics Meeting’2019. And it is with a great pleasure we see this algorithm is used in real life, for various realizations (as in this great video of GDQuest, for colorizing sprites for video games, for instance).

Scientific research carried out in a public laboratory, which becomes available for the general public, that is what we want to see!

7.3. Other news related to the G’MIC project

• Recently, a major improvement in the performances of G’MIC under Windows has been achieved, by recoding the random number generator (now reentrant)) and removing some slow mutex which were responsible of performance drops for all filters requiring sequences of random numbers (and there were many!). As a result, some filters are accelerated by a factor of four to six under Windows!

• Since December 2018, our G’MIC-Qt plug-in is available for Paint.net, a free graphic editing software application under Windows (not open-source though). This has been possible thanks to the work of Nicholas Hayes who wrote the glue code allowing the interaction between our G’MIC-Qt plug-in and the host software. Users of Paint.net are now able to benefit from the 500+ filters offered by G’MIC. This plug-in, available here, has already been voted “Best Plug-in of the Year 2018“ by the members of the Paint.net forum ☺ !

• Since October 2018, the G’MIC-Qt plug-in for GIMP has been compiled and proposed for MacOS by a new maintainer, Andrea Ferrero, who is also the main developer of the free software application Photoflow, a non-destructive image editor (more information here). Many thanks Andrea, for this wonderful contribution!

  • Since the announced shutdown of the Google+ social network, we have opened two new accounts, on Framasphere and Reddit, to share news about the project’s life (but the Twitter feed is still our most active account).

• Let us also thank Santa Claus, who kindly brang us a materialized version of our mascot “Gmicky” last year. That looks almost perfect!

  

• The G’MIC project was presented at the FENO, the “Fête de l’Excellence Normande“, from 12 to 14 April 2019, at the Caen Exhibition Centre. We were hosted on the stand of the CNRS Normandie, and we carried out demonstrations of style transfer (teaser) and automatic illumination of clip arts (teaser), for the general public.

  

• And to dig even deeper, here are some other external links we found interesting, and which mention G’MIC in one way or another:
  • A video presentation of the plug-in G’MIC-Qt, by Chris’ Tutorial;
  • The Youtube channel MyGimpTutorialChannel offers a lot of videos showing how to use G’MIC-Qt in GIMP to achieve various effects (mostly in German);
  • The Clinic, a Chilean weekly newspaper, apparently used G’MIC to achieve an effect on one of its covers (via the smoothing filter “Artistic / Dream Smoothing”);
  • Another video tutorial, showing how to use the G’MIC “Artistic / Rodilius” filter to create stylized animal photos.

8. The future

As you see, G’MIC is still an active open-source project, and with its 11 years of existence, it can be considered as mature enough to be used “in production” (whether artistic or scientific).

We have never defined and followed a precise roadmap for the project development: the functionalities come according to the needs of the developers and users (and the limited time we can devote to it!). At the moment, there is a lot of interest in image processing methods based on neural networks, and deep learning techniques. It is therefore possible that one day, some of these methods will be integrated into the software (for instance, we already have a prototyped code running in G’MIC that actually learns from image data with convolutional neural networks, but we are still at the prototyping stage…).

After 11 years of development (make it 20 years, if we include the development of the CImg library on which G’MIC is based), we have reached a point where the core of the project is, technically speaking, sufficiently well designed and stable, so as not to have to rewrite it completely in the next years. In addition, the number of features available in G’MIC already covers a large part of the traditional image processing needs.

The evolution of this project may therefore take several paths, depending on the human and material resources that we will be able to devote to it in the future (for the development, but also in project management, communication, etc.). Achieving an increase in these resources will undoubtedly be one of the major challenges of the coming years, if we want G’MIC to continue its progress (and we already have plenty of ideas for it!). Otherwise, this image processing framework might end up being just maintained in its current (and functional) state. It is of course with a hope for progression that we have recently set up the donation page. We also hope that other opportunities will soon arise to enable us to make this project more visible (you are invited to share this post if you like it!)

That’s it for now, this long post is now over, thank you for holding on until the end, you can resume normal activity! I’ll be happy to answer any questions in the comments.

---

Post-scriptum: Note that the 3D animation displayed as the teaser image for this post has been actually generated by G’MIC, via the command $ gmic x_starfield3d. An opportunity to remind that G’MIC also has its own _3D_ rendering engine capable of displaying simple objects, which is very practical for scientific visualization! We may have the occasion to talk about it again in a future post…

A special thank you for reviewing and helping to translate this article to:
Patrick David, Sébastien Fourey, Christine Porquet, Ryan Webster.

Quick digiKam Tip: Back up digikamrc file

digiKam stores the current state of the application in the ~/.config/digikamrc file. This file keeps track of pretty much everything: from the database connection profile and custom toolbar settings, to the last-used curve and sharpening parameters. So next time you install or reinstall digiKam, don’t forget to back up the digikamrc file. This way, you don’t have to configure a fresh digiKam installation from scratch. Simply copy the file to a safe location or external storage device, and drop the file into the ~/.config folder before you run digiKam.

Location Tracking for Photographers with GPS Logger and Trekarta

When it comes to Android apps for photographers, we are spoiled for choice. From depth-of-field and golden hour calculators to sun position and remote control apps – there are plenty of clever tools to choose from. But there is one particular app combination that can prove to be indispensable for any photographer on the move: a GPS logger and a GPX viewer. There are two main reasons for that.

1. Tracking your movements and saving them in the GPX format can come in handy for geotagging photos.

2. The ability to attach comments to the current location allows you to use the GPS logging app to note places you either photographed or you plan to photograph later. You can then use a GPX viewer app to see and manage bookmarked locations.

There are several apps that offer GPS logging and viewing, but you can’t go wrong with GPS Logger for Android and Trekarta. Both apps are released under an open source license, and they are available free of charge on Google Play and F-Droid.

How you set up GPS Logger for Android is a matter of personal preference. One way to go is to configure the app to automatically start tracking on boot and upload tracks to the desired destination (e.g., a NAS or a file sharing service).

Once GPS Logger for Android is running, adding a comment to the current location is as easy as pulling down the notification drawer and tapping Comment. The app saves the tracks as GPX files in the Android/data/com.mendhak.gpslogger/files directory on your Android device. To view a GPX file in Trekarta, use a file manager to navigate to the directory, and use Android’s sharing functionality to send the desired GPX file to Trekarta.

Processing a nightscape in Siril

A basic tutorial

Siril is a program for processing astronomical photographs.

In this tutorial, I’ll show you how to process a nightscape in Siril 0.9.10.

It doesn’t intend to be comprehensive tutorial but rather to present a basic general workflow that is a good starting point for those who want to learn Siril.

For this purpose, I’m sharing the raw files I used for the image I presented here, except that for this tutorial I limited the number of frames for the sake of bandwidth and processing speed.

You can find and download the raw files here (~1GB).

Setup

The raw files are placed in specific sub-folders according to their use:

• bias/offset frames → ./Bias (20 files)
• dark frames → ./Darks (15 files)
• flats field frames → ./Flats (15 files)
• main subject/light frames → ./Lights (10 files)

Bias, dark, and flat field frames are also called “calibration” frames, their purpose being to improve the quality of the image by correcting the signal-to-noise ratio (in the case of bias and dark frames) and vignetting (with the flat frames). There are several places where you can learn more about the different types of frames for astrophotography.

At the root of the folder, I placed two text files with the .ssf extension, these are scripts used by Siril for batch processing the files. Quite useful. If you want to run a script from Siril, place the .ssf files in ~/.siril/scripts. Upon restarting Siril, a new Scripts menu appears in the top menu bar, allowing you to launch the installed scripts.

I suggest you download the whole folder (~1GB), and move the scripts as indicated above. This way, if you set the working directory in Siril to the root of the folder, launching the script named processing_from_raw.ssf will automagically process the raws and create the output image in both .fit and .tif (16-bit) formats. Please note that in order to successfuly run the scripts, there must be a folder structure like the one used in this tutorial.

Step-by-step processing

I will present the steps I used to process an image of the Milky Way. I don’t know if it’s the best way, but it’s probably close to what the developers of Siril advise to do for the general case of starting from raw files (actually, I started from one of their scripts and just slightly adapted it).

We will start with processing the calibration files, and then processing the lights.

Preparing the bias frames

1. Set the working directory to the Bias sub-folder by clicking on Change dir….

2. We will use the 20 bias frames to generate a master-bias frame. To load the bias frames, click on the + button as shown (make sure that you select RAW DSLR Camera Files in the combo box) and select the bias frames located in the Bias subfolder.

3. In the “Sequence name” field, enter bias (or whatever you see fit) to set the prefix of the sequence and subsequent files, and click Convert to convert the files to the FITS format, which is the main format used by Siril. Note that you don’t need to demosaic the files yet, make sure the Debayer box is unchecked.

When done converting the bias frames, a window will pop up showing a preview of one of the bias frames. Note that since it’s not demosaiced, it will only show as a B&W channel image.

At this point, the bias frames are loaded and ready to be processed to make a master-bias frame.

4. In the Stacking tab, choose Average stacking with rejection as stacking method, and No normalisation under the normalisation combo box. You can leave the Sigma parameters at their default (unless you know or want to experiment for better values).

It should look like this:

5. Click on the Start stacking button. The resulting master-bias frame will be saved as bias_stacked.fit in the Bias subfolder.

Preparing the flat field frames

Since the flats also contain the sensor readout noise (contained in the bias frames), we should remove it by subtracting the master-bias.

1. In the File conversion tab, remove the files already loaded by clicking on the button located just below the - (minus) button, and by clicking on the + (plus) button select and load the flat frames located in the Flats subfolder.

2. Set the working directory to the Flats sub-folder by clicking on “Change dir…” and set the Sequence name as “flats”.

3. Like for the bias frames, ensure Debayer is unchecked, then click on Convert.

4. In the Pre-processing tab, check only the Use offset box, click on Browse to select the Bias/bias_stacked.fit file, and click on Start pre-processing.

5. To generate the master-flat, go to the Stacking tab, and this time set Normalisation to Multiplicative and the Stacking Method as Average with rejection.

6. Click on Start stacking to produce the pp_flat_stacked.fit master-flat frame in the Flats subfolder.

Preparing the dark frames

As with the bias and flats, you need to load the dark frames.

1. In the File conversion tab, remove the files already loaded, select and load the dark frames located in the Darks subfolder.

2. Set the working directory to the Darks sub-folder by clicking on Change dir…, and set Sequence name as darks.

3. Debayer should be unchecked.

4. Click on Convert.

5. The darks need to be stacked the same way as the bias frames. In the Stacking tab, choose Average with rejection and No normalisation.

6. Click Start Stacking.

The master-dark frame is saved as Darks/dark_stacked.fit.

Note: if you take images often in the same conditions (same air temperature, same exposure settings), you can save the dark_stacked and pp_flat_stacked files, and re-use them to process future light frames faster. I read on some forums that some astrophotographers keep their calibration files and use those for around 1 year, before taking new calibration frames.

Preparing the light frames

Now it’s time to start processing the light frames, by first subtracting the darks (which also contain the bias signal) and the flats (from which bias has already been subtracted).

1. Select the light frames in the File conversion tab.

2. Set the Sequence name to lights, and point the working directory to the Lights (Change dir...).

3. Convert the files, still without debayering.

4. Then go to the Pre-Processing tab, check Use dark, select the Darks/dark_stacked.fit file, check Use flat, and select the Flats/pp_flat_stacked.fit file.

5. Make sure that the other boxes are checked as in the following screenshot.

Note that “Cosmetic Correction” can also be done from the “Image Processing” tab.

6. Click Start pre-processing.

This will produce new FITS files with the prefix pp_light_ and the corresponding .seq file. These files are loaded.

Demosaicing the files

It’s time to demosaic our processed files. There’s something strange in the GUI, in that after pre-processing, when you uncheck “Use dark” and “Use flat” boxes, the “Debayer FITS images before saving” and the “Start pre-processing” button become grayed out.

1. In the File conversion tab, remove the selected files and load the 10 pp_light_000xx.fit files.
2. Check the Debayer box and write db_pp_light as the sequence name.
3. Click Convert.

The pre-processed lights will be saved as FITS files, and the corresponding db_pp_light.seq file loaded. Two preview windows will open this time, one with the 3 RGB channels separated, and one with the RGB composite image.

4. In the Register tab, select Global Star Alignment (deep-sky) from the registratrion method drop-down list and click Go register.

If you have more 8GB of RAM, you can try checking the Simplified Drizzle x2 box (it will up-sample the images by a factor 2, increasing the RAM usage by a factor 4). Siril will detect the stars and register each of the 10 images. The preview windows will be updated. By the way, you can play with the zoom and select AutoStretch to get a better preview of the selected image.

5. In the Stacking tab, make sure that Average with rejection is selected as the stacking method, and that Additive with scaling is set for Normalisation.

6. Click on Start stacking.

The resulting aligned and stacked image will be saved as Lights\r_db_pp_light_stacked.fit.

7. At this step, you can also save the resulting image as JPEG, TIFF, PNG, etc. for further processing in your favorite image editor. On the menu, just click on File > Save As, and pick the image format you wish (or right-click on the RGB windows and pick the format that best suits you).

Post-processing the image

Siril can do some more or less specialized post-processing to your image. I found it interesting to use.

• While the stacked image is still loaded in Siril, you can apply a log transform (it is in linear mode in Siril). I haven’t found how to do it in the GUI, but you can simply type “log” in the “Console” field at the bottom of the Output logs tab, in the main window.
• Still in the console field, you can use the command “crop” followed by the coordinates of the bounding box in pixels, to crop the image (some auto-detection tools in Siril require the image to be cropped to remove the borders introduced by aligning the images, in order to work properly). For example, my image can be cropped by typing crop 30 30 5950 3970.
• You can apply green noise removal in the “Image Processing” tab > “Remove Green Noise…”.
• Lucy-Richardson deconvolution can be applied in “Image Processing” menu option > “Deconvolution…”. 10 iterations and a Sigma value of 0.6 are a good starting point.

The resulting image can be saved as JPEG, TIFF, PNG, etc. for further processing in your favorite image editor or as a finished image if you’re satisfied.

Processing for the foreground

The problem with this whole process, is that because the images have been aligned with the stars as reference, the foreground will be blurred because earth moved between successive frames. What I do is to reprocess the light frames from just after the calibration step (i.e. after the dark and flat frames subtraction) but only skipping the stars registration step. By doing so, the foreground will undergo the same pre- and post-processing, and the resulting image will have a sharp foreground and trailing sky.

I provided a script (processing_from_raw_foreground.ssf) which will do that for you, if you already used the first script or if you use the same file naming convention as in the script.

Finally, in your favorite image editor, you can combine the “sky” and “foreground” images using a mask, to get both the sky and the foreground sharp.

Here’s what I obtained following these steps (but using the scripts), after just combining the 2 images in Gimp:

And after quick curve and saturation tweaking in Gimp:

New Topic Previews

Image previews now available for some categories

I’ve been a member of the community over at blenderartists.org (previously elysiun) for a long time (it’ll be 15 years this October according to my profile there). So it was nice to see when they finally transitioned to using Discourse a little while back.

What I really liked, though, was the work that Bart did to specific pages and tag lists to display them. Here’s what their current homepage looks like:

They use a different default main page style, “Categories” view, than we do (“Latest”). This just shows the site categories as a column on the left, then the latest posts in a column on the right.

The row of featured images along the top is actually part of a plugin that I’ll get to in a moment.

If you want to change your own default main page view of the forums, you can modify it at your account Preferences → Interface (and change it to Categories):

Fancy Category Views

The default landing page is neat, but what they did with their forum gallery page is much neater:

They set up the Topic List Previews plugin so the entire category is actually viewed as a tile of images. I think we can all agree that this is generally a much nicer way to view categories that are heavily image-based. Of course, I thought this was a natural fit for us as well!

So through the magic of having an invaluable resource like a darix, he was able to make it a reality for us!

We’ve got it implemented now on the Play Raw category (now its own sub-category under the Processing category), the Critique, and the Showcase category. If you haven’t had a chance to check it out yet, please do. (darix announced it in the thread Play raw posts and you so feel free to give us any further feedback in that topic.)

Keep in mind that for those categories, the preview image will correspond to the first image in the first post. Try to remember to make the first image in those category topics the one you’ll want in the preview.

We haven’t enabled the featured row of images after some initial feedback. We may revisit it again at some point, but hopefully the Play Raw and Showcase categories will look a little better now. It certainly makes the categories a little easier and faster to navigate now that you can see the previews directly on the page.

G'MIC Finally Accepts Donations

Help support an awesome team!

For years the incredible team over at G’MIC (GREYC’s Magic for Image Computing) have been producing an incredible image processing system and many awesome filters to go along with it. They’ve got an very active and awesome community right here on their forums and they’ve been producing all manner of neat processing filters for photographers, digital artists, and scientists.

Due to the project being under the auspices of a French Research Lab, the GREYC laboratory in Caen, France, they were limited in being able to accept any donations.

Until now!

To avoid burying the lede, go and make a donation to the fabulous folks of the G’MIC project: https://libreart.info/en/projects/gmic.

I first heard about GREYCstoration (proto-G’MIC) a long time ago (over 10 years) as the only really viable Free Software image denoising option for photographers. It allowed me to de-noise images on par (or better in many cases) with the then popular Noise Ninja. It’s been an essential part of my toolkit ever since!

Back then David Tschumperlé was really only looking for postcards from users as a “thank you” and maybe some occasional donations to pay for hot chocolates during the day. I finally got to buy him a milkshake while at the Libre Graphics Meeting 2014 in Leipzig, Germany and for the value he has provided me with his software I owe him many, many more!

Value is exactly what I want to bring up in this post. I’m sure many here have had an opportunity to use G’MIC in some form and any attempt at listing the wide range of filters and capabilities it provides would not do it justice. If you’ve realized some value from the project, now is the time to show some love.

I’ve been lucky to call David a friend for a long time now and I can personally attest to his kindness and sincerity. With that in mind I implore you: if you have a few spare dollars, euros, yen, pesos, or gold bullion please consider donating to the project and make sure the milkshakes (or hot chocolates) never stop flowing. David (and Sébastien Fourey!) are a sound investment in providing high value to Free Software artists and photographers!

Donate to the G’MIC project!

(David is too modest to really come out and ask for support but “modest” isn’t really in my vocabulary - so go donate!)

LILA

For a long time David was unable to accept donations from community members. There were some concerns from the research institution his lab is a part of in France. Just recently, though, they managed to reach an agreement where the funds flow through a French non-profit called Libre Comme L’Art, LILA, that includes the fabulous Jehan Pagès of the GIMP team as a member and where the ZeMarmot animated film is being produced.

A big “Thank You!” to Jehan and LILA for making this possible!

A Q&A with the CHDK Developers

Adding Features to a Camera: Hacker Meets Photographer

Introduction

CHDK is a free, open source software add-on that runs on Canon PowerShot cameras and expands their functionality. Some of its features are:

• Professional control: RAW files, bracketing, manual control over exposure, zebra mode, live histogram, grids, etc.
• Motion detection: Trigger exposure in response to motion, fast enough to catch lightning.
• USB remote: Simple DIY remote allows you to control your camera remotely.
• Scripting: Control CHDK and camera features using uBASIC and Lua scripts. Enables time lapse, motion detection, advanced bracketing, and more.
• PTP: Shooting control, live view, and file transfer from Linux and Windows.

I talked with the core team of developers to learn more about CHDK.

How did CHDK start? Who were the first developers? What was their role in those first steps? Do you have any information on who those people are, where they come from, or their professional background?

reyalp: That was before my time. The very first developer was VitalyB, and I don’t know much about his background.

In truth, CHDK is very loosely organized and informal, so I don’t know much about current contributors backgrounds either.

waterwingz: Before my time too. The original hack seems to have taken off in 2006/2007 when it got mention on the dpreview.com site. Many people shared random bits and pieces based on what they were personally interested in working on. There was no project organization and people did their own builds.

At some point, I believe someone with the nick GrAnd, got things organized around a wikia site and created a standardized set of build tools. Eventually an online discussion forum and autobuild server were added - not sure who gets credit for those.

But over the last 10 years, more or less, reyalp has coordinated the ongoing volunteer development efforts. There is still no real plan or schedule but there seems to be community consensus of how things get done and what gets added to the package.

Some people say that CHDK was first developed by Andrei Gratchev (here and here). I believe he is GrAnd, right? Do you know something about it? Is it possible that Andrei Gratchev is VitalyB?

reyalp: GrAnd (Andrei Gratchev) and VitalyB are definitely not the same person. They have separate accounts on assembla.com. VitalyB did the very first work on what eventually became CHDK, while GrAnd was an early developer who played a major role in organizing the project. It’s possible that GrAnd originated the name CHDK. I don’t know that history directly, but it could explain confusion over whether he was the founder.

Canon cameras run on DIGIC boards. As far as I’m aware of, at the time VitalyB did the first hack, people had already hacked DIGIC-I compact Canon cameras and could execute custom programs. What was the big leap (in terms of development or finding hooks in the DIGIC-II firmware) that VitalyB made?

reyalp: The original Digic-I cameras were actually ROM DOS running on a 16 bit x86 clone (except for S1 IS, which is VxWorks on ARM and has a partial CHDK port developed by srsa_4c).

A hack for the DOS based cameras was developed by a Czech developer.

I don’t know if VitalyB was aware of this work, but the platforms were so different there wouldn’t likely have been much overlap.

waterwingz: As far as I know, the big leap in hooking the DIGIC-II firmware was figuring out how to hack Canon’s firmware update process.

It’s stated that he hacked the firmware update process and executed his own program instead of the firmware update itself. That first program aimed to make a copy of Canon firmware. How exactly could he get a copy of the Canon firmware by blinking a LED? Why did he need a copy of Canon firmware?

waterwingz: From what I understand, he assembled a little piece of code that loaded and ran in the place of the expected firmware update code.

Once he could do that, by trial & error he learned what memory address needed to be poked to turn one of the camera’s LEDs on & off. And once he could do that, he recoded so as to dump the camera’s memory contents serially via that LED to a phototransistor interfaced to an external computer.

After that it was a matter of reverse assembly of the raw code to learn how the rest of the boot process and camera firmware worked.

reyalp: I don’t know the specifics of exactly what VitalyB did for the very first camera, but generally to make a hack work with the existing firmware, you need a copy of the firmware code to disassemble and analyze.

The advantage of using LED blinking is that the code is really simple: you just need to know how to control an LED (done writing to specific address on these cameras) and a loop.

In contrast, writing a file to the SD card requires a whole stack with an SD driver, a filesystem driver and so on.

Without having already analyzed the firmware, you don’t know how to interface with those things, and on PowerShot cameras, they aren’t really available after a firmware update file is loaded.

Blinking was used frequently in the early days of CHDK, but around 2010 Alfredo Ortega and Oren Isacson of Core Labs worked out how to run scripts in Canon’s native scripting language (which we call Canon Basic). I wrote a script to dump the original firmware from Canon Basic, and we’ve used that as the primary way of dumping firmware ever since.

Canon DSLR’s also allow this same kind of hack. Why did VitalyB start with point-and-shoot cameras?

waterwingz: I’m guessing, but probably he started there because he happened to own a PowerShot and not a DSLR. Or maybe it was just the first device that he was able to find a backdoor into.

reyalp: While Canon DSLRs and P&S use the same basic CPU and operating systems, the rest of the code is very different.

Running custom code on Canon DSLRs uses different mechanisms which weren’t figured out until much later.

My impression is VitalyB started on the camera he had (PowerShot A610?).

The lower cost of P&S also makes them more attractive to experiment with, and before the rise of smartphones, P&S were much more common than DSLRs so there was a better chance of interested developers having them.

Are there any ties between CHDK and Magic Lantern (a CHDK equivalent for Canon DSLRs)?

waterwingz: There are several people who participate in both projects and some discoveries are occasionally useful to both. But there is no coordination beyond that.

reyalp: We do share information and occasionally code, but as mentioned above, the Canon firmware up to now has been quite different. Some of the initial DSLR research took place on the CHDK forum.

CHDK was born as HDK, or Hack Development Kit, and only later the “C” was added. Regarding the idea behind the name, what exactly does it mean to say that CHDK is not a simple firmware add-on, but a development kit? Does it have to do with the capability of loading and executing custom user scripts?

waterwingz: Way before my time again - I’d only be speculating on the name’s origin. I guess the kit designation means you get the source code to do development with. I don’t think it has anything to do with user scripting capabilities.

Is this kind of hack only possible on Canon cameras? Why?

waterwingz: The exact details and mechanism will only work on Canon PowerShot cameras. They all build on how Canon supports firmware upgrades (even though CHDK does not actually modify any of the camera’s firmware).

To do something similar on a different brand of camera, you’d need to find a way to exploit any firmware update method they might provide. If there is no such mechanism, you’d need to get lucky and find some other vector.

reyalp: As waterwingz says, the specifics only apply to these cameras, but in general, most embedded devices are hackable with enough effort.

Manufacturers put varying levels of effort into preventing it, but the success of CHDK (and later Magic Lantern) involves a lot of stuff that just lined up by chance.

Another important factor in these projects is that reverse engineering is additive: the more you build and understand, the easier it is to keep up with upstream changes in new models. It’s also easier for people to add useful features, which gets more people involved and keeps the whole thing going.

Getting to that critical point on an entirely new system requires a lot of effort and/or luck.

Some of the lucky things that lined up to make CHDK take off were:

• The cameras ran VxWorks on ARM 946E-S, both of which had significant public documentation. Canon later switched to their proprietary DryOS operating system, but by that point CHDK had enough built up knowledge to carry on.
• Canon left a lot of diagnostic stuff in the code and didn’t put a lot of effort into stopping unauthorized code from running.
• There were a lot of PowerShots in circulation and they were affordable, which provided more chances of a developer with the reverse engineering skills having one.
• Canon didn’t make any effort to stop it.

Can authorized warranty repair shops refuse to service cameras because of CHDK? Does CHDK use leave any traces?

waterwingz: So far it has not been a problem as CHDK runs in RAM - it makes no permanent changes to the camera. When you turn the camera off, it disappears and you have to reload it the next time you use it.

And while Canon has made no official statements about CHDK one way or the other, there is an email somewhere from someone in Canon tech service stating that as long as CHDK did not modify the camera in any way, there was no warranty issue.

If you remove the SD card containing CHDK prior to sending your camera for service, there is really no way of anyone knowing you’ve used CHDK.

reyalp: CHDK doesn’t normally leave obvious traces, but if the camera crashes, traces of CHDK can appear in an internal crash log the Canon firmware stores in onboard flash memory.

It’s certainly possible that other traces could be present, the cameras have a lot of sub-components that could store their own diagnostic information.

I don’t recall any cases of anyone reporting having warranty service rejected for this, but that doesn’t mean it couldn’t happen.

Could you give a brief explanation of how CHDK is designed? Do you have any kind of diagram that could illustrate it?

waterwingz: CHDK is a volunteer effort and most volunteers would rather code than document in detail.

But you can learn a lot reading the For Developers section of the CHDK Wikia.

A short description is that CHDK loads by hijacking the camera’s firmware update process and then intercepts some of the camera’s RTOS tasks and replaces them with its own tasks. The CHDK tasks typically replicate the functionality of the original camera tasks but add features and functionality not included in the original Canon code.

In what language is CHDK built?

waterwingz: A combination of ARM assembler and C.

philmoz: I would add Lua to that - a lot of useful features are now in the scripts. waterwingz has built some impressive functionality with his scripts.

I suppose we should also include uBasic as we still include some testing scripts written in it. uBasic is very primitive compared to Lua, so Lua scripts are preferred.

Finally there is also the Canon Basic built into the firmware - we use this to do firmware dumping.

What were some of the difficult issues in making CHDK easier to port to new cameras?

waterwingz: Ummm… who said it was easy?

But seriously, most of CHDK is based on guesses about how the original Canon hardware and firmware works. So much of the coding was done on a “try it and see” basis.

What makes it more difficult is changes to the firmware as Canon changes RTOS’s or generations of DIGIC processors.

philmoz: The Canon firmware remained pretty stable for a while. It continued to evolve but there weren’t any real upheavals until Digic 6. This made it easier to improve the tools, and take some of the guesswork out of ports.

We have a “sig finder“ tool that analyses a firmware dump and tries to find the things needed for a port. When I first started this was pretty primitive and a lot of things had to be done manually. I spent some time improving this tool, and I think that made porting a bit quicker until Digic 6.

waterwingz also created a GUI tool for disassembling the firmware in a way that could be used in a port - I added a scripting language to this to automate the generation of some of the files needed for a port, which I think helped with new ports.

With Digic 6 the architecture changed a lot and things slowed down quite a bit. A lot of reverse engineering on the new cameras has been done by srsa_4c, reyalp, ant, and others so things are getting better again.

reyalp: One of the things that makes it difficult is how many there are. The official CHDK source supports over 150 distinct models (many with multiple firmware versions that each require distinct ports), spanning Canon releases from 2004 through 2015.

Essentially the same CHDK code runs on all of them, so if a new model does something different, you have to figure out how to accommodate it without breaking the existing cameras.

On top of that, the developers don’t own most of the cameras, so testing is difficult.

As Philmoz mentioned, with Digic 6, Canon moved to a ARMv7 architecture processor and a new display system with a TAKUMI GPU, which took a lot of work supporting. srsa_4c did much of the initial reverse engineering work, while I took the concepts from Philmoz‘s “sig finder“ and implemented them in a new tool based on the open source Capstone disassembly library.

nafraf: The scripting language developed by Philmoz helped a lot to port until Digic 5+ cameras. Using code_gen tool it was possible to port new models and improve the existing ports. On release 1.3, for example, code_gen was the key tool to add more than 60s exposure on all cameras.

In which OS do these tools run?

waterwingz: CHDK is built using the gcc compiler so I guess the tools run on anything that supports that compiler - Windows and Linux for sure.

I do all my work under Linux although I have a laptop somewhere that runs the Windows tools.

The autobuild server that rebuilds CHDK after each update and provides current downloads runs under Linux.

And there are quite a few other tools that people have created, some of which are Windows only (or using Wine under Linux) and some of which are Java based and will run on Windows, Linux, or MacOS.

philmoz: I use MacOS for CHDK development.

I also have a Linux VM I use for testing batch builds of the entire set of supported cameras, to make sure big changes don’t break the autobuild server.

reyalp: I use Windows on my primary development system, but all the core CHDK tools and build process have supported Linux for as long as I’ve been involved.

My normal CHDK development environment is MSYS shells and gvim.

I also use Linux in VMs and a Raspberry Pi for some things.

Is there any camera emulator that allows to test core code before loading it into the camera?

waterwingz: There are a couple of clever GUI emulators for testing CHDK uBASIC and Lua scripts.

But I don’t believe anyone has really succeeded in making a QEMU emulator for core code development.

All testing is done on actual cameras.

And to date, I don’t believe anyone in the dev community has bricked a camera which says something about the stability of the process!

philmoz: Magic Lantern uses QEMU to run their code in an emulator.

In theory CHDK could do this as well, but to date no-one has invested the time to create the hardware simulation bits needed.

Debugging CHDK is old school - blinking LED’s, printing messages (if you have the display working), writing log files, and lots of trial and error.

reyalp: Not emulation, but I use chdkptp a lot for interactive testing. Being able to dump bits of memory or call functions interactively from a PC console is very useful.

What’s the easiest way for someone to get involved with CHDK?

waterwingz: Download it and use it on a PowerShot.

Learn what it does and how to run scripts.

Then write some scripts on your own, or modify some existing ones.

Finally, do a port for an unsupported camera - pretty much every CHDK dev started off porting and then got hooked on doing more.

blackhole: The easiest way is to use CHDK for something creative. It’s nice to see when users show results that are the product of using CHDK. I think it’s the biggest reward for developers when they see that their work is well-used.

reyalp: The forum is the best place to get involved with the community.

To get involved with development, it really depends on your interests. If there’s something you want to add, either dive into the code or ask for suggestions on where to start.

Ports of additional cameras are always welcome too, and doing one provides a good overview of how CHDK works.

What are some tasks non-programmers can do to help the project?

waterwingz: There has been a lot of work on the CHDK wiki over the years, but there is still a ton to do.

For example, one CHDK user finds time each month to simply correct spelling and the worst of the grammar mistakes on the more popular pages and the CHDK User Manual.

reyalp: If you do something interesting with CHDK, share it in the forum. A lot of interesting projects start as riffs on something someone else explored years earlier.

Documentation always needs help, but for CHDK, a lot of it really requires careful experimentation or knowledge of the source to do well.

nafraf: If you are a CHDK user and find a bug, or a missing function on the port of your camera, please report the bug to the forum and help to test new versions.

Developers don’t have access to all models, then testing and receiving feedback from the users are necessary to help the project.

How healthy is the production of user scripts? Is it easy for a non-programmer to write a script? In what languages?

waterwingz: The huge improvements in mobile phone cameras have really impacted the market for all but the highest end or largest zoom P&S cameras.

Having said that, a core of serious photographers still work on what interests them and depend on CHDK to help capture their artistic vision.

As far as non programmers and script writing, uBASIC is about as simple as a computer language gets and there are lots of example scripts to study.

Lua provides a much richer programming environment, albeit with a bit of a learning curve.

reyalp: Easy… depends on the user.

Because CHDK is a reverse engineered hack on top of an undocumented system, many behaviors are not well specified or understood.

waterwingz improved things a lot by creating a comprehensive reference of CHDK script functions, but developing non-trivial scripts still requires significant effort and a willingness to experiment.

I’ve been using CHDK for 10+ years, and still find myself greping the CHDK source and making test cases to figure out what functions actually do.

All that said, I don’t think learning to write modest CHDK scripts is particularly harder than starting out with javascript or batch files or that sort of thing.

We should note CHDK uBASIC is based on Adam Dunkels code, not the UBASIC written by Yuji Kida for mathematical computing.

How many developers work on CHDK? How does it work?

waterwingz: There have been hundreds of contributors to CHDK over the years.

Currently there is an active core of two or three people doing original work with low level firmware stuff, a couple of people working more on the user experience, several people generating custom scripts for unique photographic opportunities, and a core of maybe ten CHDK experts not doing much coding these days but continuing to provide support to the community.

What uses are people making of CHDK? Is there any that should be highlighted?

waterwingz: There is a nice list on the main page of the CHDK Wiki - unique things like motion triggering, scripting, RAW/DNG, bracketing, and full manual exposure control.

Originally, there was a lot of interest in just getting RAW files from an inexpensive P&S camera.

More recently the focus has been on making good multi-camera rigs with full centralized control using CHDK’s PTP capability. Everything from book scanners, “bullet time” rigs, to full 3D capture for building small replicas of people. And of course the ongoing interest in time lapse videos and kite and drone photography work.

reyalp: I get a kick out of how many different things show up searching google scholar for “CHDK hack”

Currently, what is the main development effort underway? New functionality? Porting CHDK to new camera models?

waterwingz: A lot of the current development is focused on very detailed features that interest the core developers. Not really anything that will revolutionize the CHDK user experience right away unfortunately.

But there are also potentially some interesting new things in the wings if the devs working on them can ever get them finished.

reyalp: There aren’t really any major features undergoing significant development right now.

There are some ideas and experimental stuff being kicked around, like the GUI concept and some work on capturing raw outside the normal shooting process, but what eventually gets added depends on developers time and interest.

I’m trying to wrap up a few things to release CHDK 1.5 before we start major projects in the official development branch.

So there’s a new CHDK GUI underway, and it seems very interesting and more user oriented. Once finished, how will it be released to the many different camera models that already run CHDK?

waterwingz: Oh oh - you noticed that? Did you also see the comment about the dev working on it not being too good at it getting finished? ;)

For what it’s worth, there has been quite a bit of thought about keeping it generic enough so that will run on all CHDK capable cameras. That’s mostly about screen resolution issues but there will probably be other challenges. Touchscreen cameras like the PowerShot N come to mind.

What is the future of CHDK, considering the evolution of technology? How long do you think point-and-shoot cameras will stay on the market, given the rise of smartphones? Do you think the latter will replace the former?

waterwingz: As I mentioned earlier, smart phone cameras continue to improve and that has impacted the low end PowerShots that CHDK does so much for.

CHDK will continue as an interesting project as long as a people enjoy using it and creating new things with it.

blackhole: The P&S cameras with the large zoom will likely survive the competition of smartphones. Smartphones in this area will not be competitive with cameras for a long time. CHDK is likely to have a future in this area.

philmoz: I think there will also be demand for the higher end P&S cameras with larger sensors, although I don’t think the market will be huge.

The number of people with these cameras interested in CHDK is probably going to be pretty small.

Canon’s EOS-M mirrorless cameras can also run CHDK so there is some interest there.

reyalp: Low end, mass market P&S are clearly on the way out. I agree with blackhole and philmoz that higher end stuff will be around for a while to come, but the possibility of running CHDK on future cameras is always uncertain.

As I mentioned earlier, Canon DSLRs and P&S cameras have been based on different codebases, which are different enough that it doesn’t make sense to run the same hack on both.

CHDK supports the EOS M3 and M10 because they are built on the P&S codebase, while Magic Lantern does not support them.

There are signs Canon is moving to a unified codebase (likely motivated by the same market changes) in Digic 8 cameras, which may preclude CHDK as we currently know it.

However, with millions of CHDK capable P&S in circulation, there will still be potential uses for a long time to come.

Separate from smartphones, the rise of things like Rasbperry Pi, dedicated UAV and action cameras etc. have reduced cases where a hacked P&S is a clear win over other options.

In 2008, if you wanted a programmable, multi-megapixel camera with decent optics your choices were very limited and mostly expensive.

In 2019, you have a lot of options other than CHDK, but at the same time, a lot of these things can work well with CHDK too.

I think the collapse of P&S has also affected the pool of potential CHDK contributors: in 2008, a developer with a casual interest in photography would have a P&S, while today they would be more likely to have a smartphone. Someone who wants to tinker with camera software also has a lot more choices.

Could CHDK benefit from the current boom in single board computers and single board microcontrollers, like Arduino, Raspberry Pi, Esp32, Beaglebone, etc? Could those boards add even more functionality to CHDK? How?

waterwingz: Actually, there are quite a few successful projects out there using those little computers to control one or more Canon PowerShots running CHDK.

Basically anything that will support the necessary USB functionality to implement the PTP protocol and the CHDK extensions to the protocol. Applications like bookscanners, remote timelapse capture, photobooths, and multiple camera 3D image scanning.

Here is an example of Arduino and CHDK usage, with a nice PTP GUI.

reyalp: The webcams on this site are based Raspberry Pi’s using chdkptp to control CHDK cameras (chdkptp is a tool which I maintain that allows controlling CHDK cameras over USB from Linux and Windows).

One of the firsts tests with a full rig (72 cameras) shooting at same time. By nafraf, licensed under CC BY-NC 2.0.

A simple test to show the detail of a segment of the rig, how the cameras were mounted and their response after sending the turn off command using chdkptp. By nafraf, licensed under CC BY-NC 2.0.

Do you developers have time to play with CHDK? What are your preferred use?

waterwingz: There are a ton of features in CHDK and I’ve had fun playing with most of them. But for me, it’s mostly about Lua scripting when I’m actually using CHDK.

blackhole: Unfortunately my real life does not allow me to play with CHDK as much as I want.

My favorite use is when popularizing astronomy among children. It is a priceless experience when you see the glow in their eyes when they see the image of the planet they have taken themselves. For me, this is the highest value of CHDK.

reyalp: I use CHDK raw for general shooting, and scripts to do timelapse.

I’ve used motion detection for lightning and fireworks.

I also do some lo-fi astrophotography using CHDK scripts to take lots of exposures to guide and stack in software.

But what I do most is take test shots of my desk while working on the code ;)

When did you joined CHDK as a developer? Why? What is your background? What is your role?

waterwingz: July 2010 according to the records on the CHDK forum. But it seems like it was only nine years ago.

I got started because it combined two of my hobbies - computers and cameras.

Since then I’ve ported several cameras, contributed some original code, helped fix some bugs, coded many scripts, written a lot of documentation on the wiki, and helped a few newbies on the CHDK forum.

philmoz: I started with the G12 port in November 2010.

I’ve been a software developer for nearly 40 years and like waterwingz, photography is a hobby. CHDK looked like fun and I wanted something to keep me programming - my day job was more management than development.

I was pretty active until 2016, when real-life got in the way.

I now do mobile app development full time, so don’t spend much time on CHDK coding these days.

blackhole: I joined in August 2010.

Prior to that I was just reading the forum as a guest and using CHDK on the old A530 and A590 cameras. At that time I was looking for a better solution for cheap-modified webcams, which were then popular in amateur astronomy. The logical solution was to switch to something cheap with a CCD sensor, so the decision fell on Canon cameras and CHDK.

I became interested in the programming, so I started to collect knowledge on the forum and in the end I made my first port.

reyalp: Around 2008, I happened to get a Canon A540 and google “firmware hack” or something like that just for kicks. I had a background in C and assembly, and a somewhat neglected interest in photography going back to film days, so it seemed like a fun thing to play with.

For me, CHDK development was a nice change of pace, a throwback to the early PC days where if you want to draw something, you write directly to video memory instead of going through a bunch of APIs.

As people came and went from the project I somehow ended up being the chief cat herder.

nafraf: I started in June 2012. My first port was A810. I was using CHDK for a multi camera project and it was difficult to find ports of recent cameras.

With CHDK, the exposure time and ISO values can change in 1/96 EV steps. The first part of this video was made using a standard timer with exposure changes of 1/3 EV steps. The second part was made using script isoinc.lua, no post-processing. By c_joerg, licensed under CC BY-NC 2.0.

Is there anything that you would like to add?

waterwingz: Getting involved in CHDK is a bit of a trap. Once you get in, your free time just disappears. But it can be a lot of fun!

philmoz: The people who have worked on CHDK over the years are an amazingly talented, fun, and helpful group. I have learned a lot from this project and really appreciate the willingness to help, and assistance I’ve received.

reyalp: I’d like to thank all the people who have contributed over the years, and Canon for turning a blind eye to it for so long.

blackhole: CHDK is a very fun and creative project. I invite all photographers and programmers to join us and express their creativity through this project and share their experiences with us. In the end, I would like to thank the entire CHDK community for a pleasant companionship for the last ten years.

nafraf: Thanks to all people involved with this project. I have learned a lot during these years.

This video was created by changing the zoom levels between 24mm and 1200mm (35mm). For each zoom level, 2 images were taken at a time. The camera was a Canon SX50 with 200 zoom levels and CHDK. A special script was used. By c_joerg, licensed under CC BY-NC 2.0.

Thank You CHDK devs!

I want to thank CHDK devs again for taking the time and being patient enough to chat with us, as well as sharing images of their CHDK use!

I also want to thank CHDK users Garry George, Peter Laudanski, Andrew Hazelden, c_joerg, and keoeeit for having kindly shared some images and answered questions about how they shoot them!

Finally, I want to thank Pixls members paperdigits and afre for their invaluable support, without which this interview wouldn’t have been possible.

The CHDK community gathers around https://chdk.setepontos.com/ and all official CHDK documentation can be found on http://chdk.wikia.com/wiki/CHDK.

Libre Graphics Meeting 2019

Let's participate!

It’s that time of year again: Libre Graphics Meeting 2019 is fast approaching!

This year the meeting will be May 29 to June 2 in Saarbrücken, Germany. This is extra exciting because Saarbrücken is centrally located enough that we should have a nice representation from projects and community members. Members of both RawTherapee and darktable live nearby and will be in attendance (along with others from those projects and many others).

Participate!

I’m hoping to have a good representation this year, so first and foremost - please, please consider participating by giving a presentation, leading a workshop, or even a quick lightning talk! The Call for Participation page is here:

https://libregraphicsmeeting.org/2019/call-for-participation/

I will make myself available to help in any way I can. If you want a hand with the presentation, design, graphics or whatever please feel free to ping me (also - remember that we try to archive all of our presentations and material in our Github repo so you can grab any of the assets from there as well)! This is a great opportunity to spread the word about what we’re up to and the many, many awesome projects everyone has created, maintained, and contributed to for Free Software photography.

The deadline for submittal is coming up on January 15^th!
If you think you’d like to present or host a workshop please submit as soon as possible.

Cheers!

I am particularly excited about this meeting because a) I don’t get to attend every year so this is the first time I’m able to make it in a few years, and b) this is a great opportunity to really get a bunch of community members to come together! Saarbrücken is on the high-speed rail network so it’s readily accessible from many places (now you have less excuses to not make it).

I love hanging out with y’all. It’s great to nerd out about photography and catch up. Sometimes it really helps to be able to speak face-to-face and this is the perfect opportunity to also be exposed to all manner of other Free Software projects (or to expose ourselves to others?).

Besides, how else am I going to capture some fun photos of y’all?

What’s Going On

We have a few things planned for sure at the meeting and you’re going to be really sad if you miss them by not coming!

1. PIXLS.US BoF (Birds of a Feather)
   This is a special session set aside for a couple of hours for the entire community to get together and chat about what’s going on, what we’d like to do, and what’s coming up.
2. Photowalk
   I’ve been doing photowalks every year that I’ve been because this is almost the ultimate way for me to spend time with folks (unless we can do a beer drinking photowalk all at the same time…).
3. PIXLS.US Presentation/Update
   I’ll (We’ll? Anyone from the community is more than welcome to help me present on this) present on the community and what we’ve done so far and what we’d like to accomplish moving forward. This is our primary way of reporting out to the wider community who we are and what we’re doing.
4. State of the LGM
   This is a couple of slides that are included at the beginning of the program giving an overview of the state of the entire libre graphics ecosystem.

I really, really, really want to be able to add to this list with other presentations (or lightning talks, etc) that the community will give! I may even submit a couple of presentations to talk not just about the community but maybe about our technical work coordinating the forum and providing services for the projects (as well as invite other related projects to come join us).

We can make this an incredible and memorable meeting with a fantastic opportunity to meet friends in person and have a wonderful time!

Goodbye Google Analytics

A little less tracking for the new year

Over on my personal website I decided to stop using third party trackers and assets to keep from exposing visitors to unintended tracking. Third party assets expose a user to being tracked and analyzed by those third (or fourth, or more) parties and honestly this is something the web could use a little (lot) less of. I loved having stats early on when we started this crazy idea for a community and as I mentioned on my blog post, it’s a Faustian bargain to get stats at the expense of allowing Google to track what all the users of the site are doing. No thanks.

I figure it’s the eve of a new year so why not start it out right and reduce the tracking footprint of the site?

This all started by noticing that some new browser feature strips referer information from requests (thanks @darix) and we were using them to target specific areas of websites that we manage comments for. It came to my attention when I was reading the release announcement for digiKam 6.0.0 beta 3.

While fixing that problem, I found that once we fixed the referer requirement problem I was still seeing issues with Privacy Badger blocking our embed code. On further inspection it boiled down to using Google Analytics on our base domain (pixls.us) and having a cookie set by Google, which then got sent with embed requests from other websites (digiKam and darktable). This triggered the heuristic blocking by Privacy Badger.

Honestly, we derive very little value from the analytics for the price (privacy) we pay to use it. Better to simply remove it.

We still do analytics but we own the stack ourselves (thank you so much andabata!). If you want to block our own analytics the domain is: piwik.pixls.us.