Run a virtual conference using only open source tools mairin Thu, 04/27/2023 - 03:00

Comments are closed.

These comments are closed.

3 key open source challenges in developing countries Ahmed Sobeh Thu, 04/27/2023 - 03:00

Comments are closed.

These comments are closed.

Test your Drupal website with Cypress cobadger Wed, 04/26/2023 - 03:00

Comments are closed.

These comments are closed.

5 open ways to help UX designers and developers collaborate better kriker Wed, 04/26/2023 - 03:00

Comments are closed.

These comments are closed.

What's new in GNOME 44? Jim Hall Tue, 04/25/2023 - 03:00

Comments are closed.

These comments are closed.

Darktable 3:RGB or Lab? Which Modules? Help!

Original post in French by Aurélien PIERRE, edited by the pixls community.

Darktable is slowly converging to a scene-referred RGB workflow. Why is that? What does it involve? How does the use of darktable change? Answers here…

This article begins with a 3 section introduction of the Lab space. You don’t need to understand it in detail in order to understand what happens next.

What is Lab?

The color space CIE Lab was published in 1976 by the International Commission on Illumination (CIE), in an attempt to mathematically describe the color perception of the average human being. Lab space aims to decouple the brightness information (L channel) from the chroma information (channels a and b) and takes into account the non-linear corrections that the human brain makes to the linear signal it receives from the retina. Lab space is derived from CIE XYZ space, which represents the physiological response of 3 of the 4 types of photo-sensitive cells in the retina (the cones).

The XYZ space represents what happens in the retina, and Lab represents what subsequently happens in the brain, but both color spaces are models, that is, attempts to describe reality and not the reality itself. There are always discrepancies between a model and reality, but these models are refined and improved as research progresses. Moreover, a model often represents reality only under certain conditions and assumptions, which define the area of validity of each model.

Regarding their respective areas of validity, XYZ works well almost all the time, Lab only works as long as the image has a contrast less than 100:1 (i.e. a maximum dynamic range of 6.5 EV). In the context of the creation of the Lab model in 1976, researchers were working with scanned negatives, and color negatives have a dynamic range of 6 to 7 EV. 6.5 EV is also the static contrast of the retina, and it was a little after 1976 that we realized that the brain was constantly performing HDR fusion of several images per second, meaning that static contrast as a model parameter doesn’t make much sense in the context of human vision.

What is CIE Lab for? It is intended to predict the perceptual difference between 2 colors (the delta E) and to make gamut adaptations when converting an image from one color space to another. One can then try to remap the gamut to the closest color in the target color space via strategies that minimize the delta E digitally.

The big disadvantages of Lab are:

1. It doesn’t work well for strong contrast (> 7 EV), and especially outside the range [1:100] Cd/m²,
2. It is not linear in hue, i.e. if one fixes a pixel’s a and b chromaticity components and changes only its brightness L, the same hue would be expected at a different brightness (this was the design purpose of the Lab space), however there is a slight shift in the hue, more or less marked depending on the original color of the pixel.

What is Lab doing in darktable?

The original idea was to allow separate manipulation of the brightness and chromaticity. In 2009, the year of the project’s creation, cameras had dynamic ranges quite close to Lab’s valid range; the idea was far from bad at the time, especially because darktable did not have a complex masking option then.

Advantages:

1. Lab, being a reference space and therefore independent of the display color, makes presets very easy to set up and transfer,
2. Lab sets the middle gray (18%) to 50%, so the interface is more intuitive (the middle gray is in the middle of the graph of the tones, for example).

Problems:

1. Today’s cameras have dynamic ranges that are largely outside of the conditions under which Lab is valid, which makes the defects of this space more apparent. With dynamic ranges from 10 to 14 EV at 100 ISO, any recent camera does HDR by default, and Lab is not designed to handle that much dynamic range
2. Pushing pixels in Lab space is very risky, especially when tackling compositing and image fusion with softened and feathered masks. We’ll get back to that, but it has to do with the next problem…
3. Lab is not adapted to physically realistic corrections, such as blurring, deblurring, denoising, and any filter that simulates or corrects for an optical effect.

In brief, Lab was a youthful mistake. That said, all other photo processing pieces of software seem to work by default in non-linear RGB spaces (with a “gamma” applied at the beginning of the pipe) that are basically equivalent (regarding their flaws and drawbacks for image filters).

How does Lab work?

Everything (e.g. the camera sensor) starts from a linear RGB space. We convert linear RGB to XYZ. For the purposes of the demonstration, we can consider the XYZ space as a special RGB space whose primary colors have been slightly manipulated (that’s not the case, but it behaves the same way). XYZ is also a linear space.

We then switch from XYZ to Lab by applying a “gamma correction” on the luminance channel (from Y to L), and a rotation on the channels a and b. Mathematically, Lab is like applying 2.44 gamma to linear RGB – it poses the same practical problem: it’s highly non-linear.

Summary

Lab doesn’t work for high-contrast images and doesn’t work well for images with moderate contrast. It encodes pixel values in a perceptual manner rather than physical one, which will pose a problem in the following. Lab was not designed for image processing, but only as a way to study human vision.

Precision: I have used the term “gamma” or “gamma correction” incorrectly here. Strictly, a gamma function is the specific (technical) electrico-optical transfer function (EOTF) of old-school CRT screens, which is a power function with an exponent between 1.8 and 2.2. Nowadays, people incorrectly name “gamma” any power function used for technical integers encoding or artistic lightness adjustments, which is confusing. Any encoding transfer function (using a power function or not) should be called OETF (Opto Electrical Transfer Function), and is used only to alleviate the limits of 8 bits integer file formats. Any artistic power-like brightness corrections should be called a tone curve. Even if the operation is the same, it does not have the same meaning and should not be applied at the same place in the graphics pipe. But ICC nomenclature continues to call “gamma” the exponent used to encode/decode RGB pixels when using integer file formats, so here we are, mixing unrelated concepts under an umbrella name just because the maths write the same. But, when communicating with people out of the industry, it’s often easier to use the incorrect name so that everyone sort-of understands, even if it carries on the confusion.
By the way, power-like OETF are completely unnecessary as long as you use floating point arithmetic and files format (32 bits TIFF, PFM, OpenEXR…).

The limits of non-linear spaces in image processing

First of all, what do we mean by “linear”? If y is linear with respect to x, it means there’s a relationship between x and y in the form y= a . x + b, where a and b are real constants. Linear means proportional to something plus or minus a constant.

So, when we talk about linear RGB space, we mean that the RGB values are proportional to something. But proportional to what?

The sensor counts the number of photons it receives at each photosite. Every pixel contains information on the light spectrum captured at its position, in the form of 3 intensities (red, green, blue). The coefficient of proportionality a between the number of photons and the final RGB value is the ISO sensitivity of the sensor. The constant b is the sensor noise threshold. The RGB signal is proportional to the energy of the light emission picked up by the camera sensor.

From the point of view of human perception, these intensities being proportional to the physical energy level of the light emission, does not make sense. In fact, the brain applies a non-linear, logarithmic correction that the Lab color space approximates using a cubic root. This means that we have an increased sensitivity to dim light, and reduced sensitivity to bright light.

However, all optical operations that are performed during image capture (e.g. lens blur, noise creation, or the effect of a color filter added to the lens) are applied directly to the photons. To reverse the lens blur or to simulate it when processing, we need to work on the linear RGB information, which is the closest thing to the photon data that is available to us.

See for yourself: Which one of these two computer generated bokeh (original below) seems the most natural to you? (See also a more spectacular example on Chris Brejon’s website)

Observe in particular how the dark silhouettes (bottom left) merge into the light background, or the contrast of the pentagons formed by the lens diaphragm on spotlights.

Another example, with a simple blur on smooth surfaces: Which of these gradations seems to you to be the most progressive?

These two examples were generated with Krita, which allows you to work in both linear and non-linear RGB, and has filter layers including a physically realistic lens blur.

This type of problem will occur the same way in darktable, as soon as you use the modules sharpen, high-pass, low-pass, and feathering/smoothing of drawn and/or parametric masks (which are blurs).

Blurring, deblurring, or anything else connected to optics must take place in linear RGB. There’s no mathematical model* that allows correct gradients in RGB encoded for display (with an OETF) or in Lab, due to loss of connection between pixel values and light energy.

* and just because the problems aren’t visible all the time doesn’t mean the problems aren’t always there. We can, up to a certain point, hide them with mathematical trickery (thresholds, opacity, etc.), but they will always end up coming out at the worst time. Trust me, I know exactly where to push to make it break.

This is also the problem that arises with hue zones blending in the color zones module (even if a tweak, introduced under the “smooth” process mode, attempts to hide this under the rug), which produces granular and sharp transitions.

The only darktable module that works in Lab to make a blur, and where it still works reasonably, is the local laplacian mode of the local contrast module. The price we pay for it to work is that it’s very computationally heavy and theory is like rocket science. And, even if the blur is stable, it comes with an ungracious desaturation with a hue shift to muddy grey-blue when you push the sliders a little too hard.

The benefits of a linear RGB treatment

So here’s where you say “as long as I’m not blurring my images or working only on color, I can still use Lab”.

That’s partly true, but in fact, even in those cases, working in linear RGB is simpler, with faster algorithms that can tolerate more extreme adjustments without showing annoying side-effects. Also, once again, Lab can’t support high dynamic ranges, so care must be taken to use the Lab modules after HDR tone mapping.

Strictly speaking, the only application where Lab is required is the gamut mapping, when changing color space before sending the image to a file or to the screen. And even then, since 1976, better spaces have been developed (IPT-HDR, JzAzBz) for this purpose, in HDR and with an almost perfect linearity of hues.

The current state of darktable

With the release of darktable 3.0, the default pipeline (i.e. the basic module order) has been reordered around filmic RGB. There are 4 essential steps in this pipe:

1. the demosaic module, which converts the raw file (which only contains the intensity of a single layer, R, G or B at each pixel site) to a picture (with complete RGB data for each pixel location),
2. the input color profile module, which converts the sensor’s RGB space to a standard working color space,
3. the filmic RGB (or the base curve) module, which translates between linear space (proportional to light energy) into non-linear (perceptually compressed) space,
4. the output color profile module, which converts from the standard working space to the RGB space of the screen or the image file.

Note that the base curve approach remains the one applied by default because it allows darktable to more-or-less approximate the rendering of the camera JPEG as soon as the software is opened, which seems to be the preference of many users. Nevertheless, as part of darktable 3.0, the base curve was pushed back in the pixel pipe by default, to just before the filmic RGB module, which makes it safe for the colors produced by the modules that are applied earlier. The base curve module also was provided with a color preservation mode, which produces results similar to filmic RGB. Between base curve and filmic RGB, for darktable 3.0, the difference is now only about ergonomics and on the ability to recover very low light. filmic RGB is a little more complex to understand but faster to set up (once properly understood), and is more powerful when working in deep shadows.

Modules that work in linear RGB and output in linear (thus leaving the pipeline linear after them) are:

1. exposure
2. white balance
3. channel mixer
4. tone equalizer (which is linear in parts).

The advantage of performing linear operations is that they do not affect the chrominance of the image (because changing the luminosity leaves the chrominance intact) and preserve the energy proportionality of the signal. These modules must be positioned before filmic RGB or the base curve. Exposure and tone equalizer are recommended prior to the input color profile. They can be used safely and without moderation. Note that there is a catch here on the tone equalizer, which preserves local linearity (within the image areas), but not the overall linearity (between zones). It corresponds to what would happen if we walked onto the scene with a flashlight, and hand reilluminated the objects in the scene, so we still keep the physical coherence of the signal.

Modules that work in linear RGB and carry out non-linear, but chrominance-preserving operations, (provided that the chroma preservation mode is activated) are:

1. RGB curves
2. RGB levels

The chrominance is preserved via methods that constrain the RGB ratios in and out of the module, so as to keep them identical. Note that RGB curves and RGB levels can be moved before or after filmic RGB depending on the intention, since they’re doing non-linear operations anyway. On the other hand, be careful not to use the mask feathering on modules that come later, as linearity is no longer assured and mask blurring + blending could produce unpleasant results.

Modules that work in linear RGB and carry out non-linear operations without preserving the chrominance are:

1. local tone mapping (we’ll get back to that)
2. color balance
3. LUT 3D

Color balance is designed to be applied to linear RGB data that hasn’t been corrected for contrast, i.e. before filmic RGB, tone curves etc. It does not preserve the chrominance because its explicit purpose is to adjust chrominance creatively. Similarly for LUT 3D, for which the main goal is to emulate analog film emulsions or complex aesthetic transforms.

I remind readers here that filmic RGB is a dynamic range compressor, from the high dynamic range of the camera to the low dynamic range of the screen. It is not a tone curve intended to apply an artistic correction, but a mapping of tones to force fit the sensor data into the the available screen space. filmic RGB tries to protect the details as much as possible (which we assume a priori are in the middle tones) and to keep a certain optical readability in the image.

Before filmic RGB, in the linear pipe, we still find some modules that work in Lab but perform linear operations that should (strictly speaking) be realized in linear RGB:

1. contrast equalizer
2. high pass
3. low pass
4. sharpen
5. denoise (non-local means)

These modules need to be adapted in the future to be able to work on a linear Yxy space (derived from CIE XYZ) because it is a mistake to make them work in Lab (at least, as a default). It’s a relatively easy job to do, because Yxy breaks down the luminance (Y channel) and chrominance (channels x and y) with a logic similar to Lab, minus the non-linear transformation. In the meantime, you can continue to use them, but with moderation. For the contrast equalizer, note that it uses an edge-sensitive wavelet separation, which makes it quite cumbersome to execute, but very effective at preventing halos, even considering that it works in Lab.

After filmic RGB, in the non-linear pipe, are all the other Lab modules, since they require low dynamic range. Some of these modules could also be converted to xyY and moved before filmic RGB in the future (in particular the soften, grain and fill light modules). Also note that the vignette module was left at the end of the pipe, as before, even though it works in RGB. It’s likely it’ll be better off before filmic RGB, or even before the input profile, but its code is surprisingly complex for what it does, and I haven’t had the time to unravel the imbroglio in order to understand what its working hypotheses are.

Modules not recommended

A number of modules are not recommended due to fundamental errors in design (based on my personal opinion, which is based on my practical and theoretical experience in image retouching), and in the spirit of streamlining the workflow with a minimum number of steps. There is nothing stopping you from continuing to use them, especially since users regularly introduce me to new use cases that I hadn’t thought of. But the idea here is to give you the keys to the best possible result as quickly as possible with as little fuss as possible.

Local Tone Mapping

Local tone mapping internally encodes RGB values logarithmically (they are then decoded at the output, so no problem at at this level), then applies a bilateral blur to these logarithmic values. As we saw above, theory is clear: a blur, on anything non-linear, produces halos and fringes. And as promised, the default setting range of this module is much reduced, so that users have become accustomed to merging the output of the module with low opacity – this is only hiding the misery.

Prefer the tone equalizer.

Global Tone Mapping

This module works in Lab color space to perform HDR compression, and if you have followed my explanations, you will understand that this is a contradiction in terms. In addition – and this is important – the white value is adjusted automatically from the maximum in the image, so the overall brightness of the image may change depending on the size of the export, due to the smoothing effect of the setting to scale (interpolation). To be expected: a lighter or darker JPEG than the preview in the darkroom.

Prefer filmic RGB.

Shadows and highlights

Similarly, this module works in Lab color space to perform HDR compression and uses a Gaussian or bilateral blur to isolate highlights and shadows. In practice, it gives halos quickly as soon as you push the parameters (even if the bilateral blurring lessens the problems a little), and it even tends to add local contrast (as a secondary effect) in the highlights, giving clouds a very HDR look. In the shadows, used a little hard, colors turn blue-grey. In practice, it does not work, except for minor corrections.

Prefer the tone equalizer.

Low-pass filter

The low-pass filter is actually a simple blur. A lot of people use it to invert the contrast, and then blend it with overlay or soft/hard/linear light, to compress the dynamic range. This is in fact exactly what the shadows and highlights module already does in fewer steps for the user. As mentioned above, the low-pass module works in Lab color space, so for the blur… Expect the worst.

Prefer the contrast equalizer for blur, or the tone equalizer for local dynamic range compression

High-pass filter

A lot of people use the high-pass module by blending it with overlay or soft/hard/linear light, for adding sharpness. This is in fact exactly what the sharpen module already does. The high pass is achieved by subtracting between a blur (low-pass) and the original image, so we have the same problem as for the low-pass because it’s still working in Lab.

Prefer the contrast equalizer for fine sharpness, or the local contrast for the general sharpness.

Sharpen

The sharpen module was originally intended for sensors with an optical low-pass filter as well as the smoothing due to demosaicing in some cases. First, as this module works in Lab, you need to avoid pushing it so much that it produces halos. Second, the internal sharpening method (using unsharp mask) is rather archaic and quickly artificial, even in RGB mode. Thirdly, in view of the sharpness of modern optics, given that many sensors no longer have low-pass filters, and that most of the photos will be exported at a reduction ratio of at least 8:1 (24 Mpx sensors to 3 Mpx screen), pixel-level sharpness enhancement has become practically useless. Generally speaking, the digital photographer of the 21st century would benefit from calming down with the crisp sharpness - it would be good for everyone.

Prefer to the contrast equalizer to deflect the optics via the presets provided, or the local contrast for general sharpness.

Monochrome

The monochrome module works in Lab, which it uses to define a weighted contribution of certain colours to the density of the black, in order to convert color into shades of gray. The problem is that the interface is quite sensitive to the settings, and a small correction can produce large changes and break the overall contrast in a rather ungraceful way. In practice, getting a predictable result is quite difficult and this module often results in a lot of tedious micro-adjustment sessions.

The idea of a weighted contribution of colors to the density of black comes from silver film, which behaves exactly the same way as this. But, as you saw coming, film doesn’t work in Lab and is not perceptually realistic. This idea is taken up in a physically realistic way in the channel mixer module, where several emulsion presets of commercial silver film are offered to create a grey channel. Note that, in order for the coefficients to be accurate, the colour space of the operating mode (in the module input profile) must be set to REC 709 linear, otherwise the settings will have to be adjusted.

For a black and white treatment that is based on human perceptual luminance (linear), simply lower the input or output saturation to 0% in the color balance module (right-click on the slider and enter 0 on the keypad – the setting is only up to 50% by default in the interface).

Prefer the channel mixer for a silver approach or the color balance for a perceptual approach.

Fill light/Bloom/Zone System

These three modules aim to re-illuminate a part of the image, and attempt to dilute the correction in intensity and in space by blurring in the picture. But since they’re working in the Lab color space …I won’t say it again… The results are just bad all the time, except with very soft settings, in which case you didn’t’ really need those modules in the first place.

Prefer the exposure module with masks, or the tone equalizer

Color Correction

Every photograph has at least two sources of light: a direct source (lamp, sun, candle) and a reflected source (walls, clouds, floors, ceiling). It often happens that the white balance of these two sources does not coincide. In practice, human vision has ways to correct for this this, but not the camera. So it requires a separate white balance correction for the highlights (which generally receive direct light) and the shadows (which usually receive reflected light).

This is what the color correction module offers you, again in Lab color space, and with mixed and unnatural results as soon as you push the adjustment. When you think about it carefully, the white balance can be reduced to discussions of light spectrum, and the correction is simpler in RGB, especially to manage progressively of correction.

The color balance module allows you to adjust this quickly, and not just for the shadows and the highlights, but also for midtones. Using the color-pickers, to the right of the tint sliders, it also allows you to go directly to sample neutral tones in the image (for black, gray and white) and let the software calculate the complementary color. See the manual for more details.

Prefer color balance.

Velvia

Velvia works in RGB and works on a logic quite similar to the color balance saturation. On the surface, it smells good. Except that in fact, its colorimetric equation is not perceptually correct. What it’s doing is changing the saturation (which is its intention), but at the same time it also changes the hue and brightness (which becomes awkward). The problem is that it seems to have been optimized for non-linear RGB. As a result, it is the kind of module that is typically unpredictable.

Prefer color balance.

Levels/RGB Levels

These two are working as they should, no problem with that. But when you look at the code, you can see that it duplicates exactly the slope/offset/power mode of the color balance module. The white point is scaled by a simple exposure correction, such as the slope factor or even the exposure of the exposure module. The black point is adjusted by adding a constant, such as the factor of the offset, or the black level correction of the exposure module. The grey point is adjusted by a power function (sometimes improperly called gamma), just like the power factor of the color balance. They are not just the same features, they are exactly the same math. The difference is therefore not only in ergonomics, but also in the fact that the color balance gives you the numerical value settings, making them more easily transferable from one image to another or from one application to another. Curves and levels also assume you work SDR images, with data encoded between 0 and 1. If you work HDR pictures or raised the exposure quite a lot earlier in the pipe, the pixel values will not be clipped, but the GUI will not give you control over the pixels above 1 (or 100 %).

If you already use the color balance, there is no need to add an additional level module. Finish your retouching in the same module.

Curves/RGB Curves

These also work well, but considering their classic use … are they really useful? Usually they are used to add/remove brightness, which falls in the same use case as the grey of the levels module or the power of the color balance module, or to add/remove contrast, which can be adjusted or by decreasing/increasing the interval between white and black (in a linear way) or by applying a non-linear brightness compression, again available from color balance.

Curve ergonomics is a real problem in an RGB linear workflow, because the middle gray is assumed to be in the center of the graph, which therefore assumes that we are working in non-linear RGB (where the gray at has been increased to 50%). In a linear encoding, the standard medium grey is expected at 18% (but the practice depends or where you anchored your exposure in camera), and the contrast control around this value not being centered on the graph becomes complex in the interface. In addition, the graph of the curves assumes a limited RGB signal between the values 0 and 100% (or 1)… 100% of What? White screen luminance. In a linear workflow, the HDR signal can go from 0 to infinity, and it is at filmic RGB step that we’re in charge of putting everything back between 0 and 100% of the white screen.

The contrast in the color balance module is compatible with this approach using the contrast fulcrum parameter, which allows the selection of the contrast reference. Thus when changing the contrast, we increase the light above the fulcrum, and reduce it below, but the fulcrum remains unchanged. The display workflow (in Lab or non-linear RGB) always has the implicit assumption that gray is 50%, uses it as a contrast reference, and doesn’t allow you to change that value.

Prefer color balance.

Contrast/Brightness/Saturation

Module working in Lab, which duplicates again the modules levels, curves, and color balance while adding undesirable effects on colours.

Prefer color balance.

Modules to be used with care

There is no correct replacement for the following modules for the moment, but they should be used with caution because they can be unpredictable and can cause you to lose a lot of time.

Vibrance

Vibrance works in Lab by applying a saturation correction that penalizes already saturated pixels to avoid over-saturation, but also tends to darken colors. The result is far from ugly, but the problem is that we can’t control how much we darken for the amount we resaturate.

Prefer color zones with a selection by saturation.

Color zones

This module would be awesome if the merging of colour zones were more progressive. It now has two processing modes (strong, the old, and smooth, the new) which are trying to meet this challenge of two different ways, resulting in transitions too discrete for the new, and too abrupt for the old. One more time, it works in Lab, when similar functionality in Capture One seems to be using HSL or HSV, which seems to perform better than Lab.

In some cases, color zones will benefit from being replaced by the color balance module where parametric masking may be used to isolate the shades you want to act on. Then the refinement of the guided filter parametric mask should help in difficult cases. For the rest, color balance allows us to change the shade, saturation and brightness exactly the same.

Note, however, that the color balance module, although working in RGB internally, merges the masks into Lab because this module is older than the possibility to have 100 % RGB modules, and converts from Lab to RGB internally. We’re still working on it…

Prefer color balance.

Vignetting

Adding a vignette around an image is not complicated: you just have to to gradually lower the exposure, and eventually the saturation with a drawn mask. However, the vignetting module performs incomprehensible black magic, which is much more complicated than that, with an internal homogenization which would be superfluous if things were well done. The result is rarely natural, the transition in luminosity being too violent compared to a real vignette.

You will get better results with an instance of the exposure module set to -0.5 EV, a circular mask with a large transition area whose polarity is reversed, possibly coupled with a desaturation in color balance to which you pass the same mask as used in exposure (via a rasterized mask).

Prefer the exposure (and, optionally, the color balance saturation) modules.

Mask blend modes not recommended.

Few people know this, but the blend modes lighten, darken, overlay, soft light, hard light, pin light and linear light implicitly expect the grey level to be 50% grey and are thus totally connected to the display-referred workflow. The blend modes are going to treat the pixels differently depending on whether they are above or below 50 %. Remember that the linear RGB workflow keeps the gray point at 18% (or even less). These blend modes will therefore behave in a way that is unpredictable in the scene-linear portions of the pipe.

In linear RGB, you should only use blend modes based on arithmetic operations (addition, multiplication, division, subtraction, average), on maximum/minimum comparisons (screen) or on channel separations (hue, color, chroma, etc.).

Note that the multiply mode is one of the most powerful in linear RGB. For example, to enhance the contrast of an image in a natural way, it is enough to use an instance of the exposure module blended with multiply. Set the exposure between 2 and 3 EV and the opacity between 10% and 50%. Exposure is then used to control the pivot of contrast, and opacity the intensity of the effect. It’s fast, simple and effective.

A minimal workflow for beginners

In darktable, you can choose between many modules that allow you to do the same thing in a lot of different ways. But this is merely an illusion of choice, as many of them have more disadvantages than advantages (provided you want to achieve predictable results for demanding edits). If you open the code for any of the modules not recommended above, you will see that they are almost all dated 2010-2011 - the only reason we retained them was to maintain compatibility with edits performed in prior versions of darktable.

You can perform at least 80% of your processing with just 4 modules :

1. exposure
2. white balance
3. color balance
4. filmic RGB

The reason they’re so powerful is because they’re actually extremely simple, when you look at their equations:

• Exposure: RGB_output = exposure × RGB_input + black level
• Color balance :
  • Slope/Offset/Power: RGB_output = (slope × RGB_input + offset)^power
  • Contrast: RGB_output = (RGB_input / pivot)^(contrast × pivot)
• White balance: RGB_out = coefficients × RGB_in
• filmic RGB is a little more complex, but it’s still high-school level math

With these 4 modules, you have everything you need to produce a correct image in terms of colorimetry, contrast, and artistic intent. Remember to turn off the base curve if you use the filmic RGB module. Then, if needed, finalize your edit with the following modules:

• To improve sharpness, the best option is the local contrast module in local laplacian mode
• To deblur the lens, you have deblur presets, more or less pronounced in the contrast equalizer
• To denoise, the best algorithm is in the denoise (profiled) module. Use non-local means auto mode if you don’t want to break your head
• To remove haze, you have haze removal
• To convert to black and white, the easiest way is to use the film presets in the channel mixer
• For creative control of overall contrast and re-lighting of the scene a posteriori, use the tone equalizer module

Some of the following modules have an underestimated power, and they are vastly underutilized:

1. The exposure module, with its masks, can replace all the the other methods of mapping HDR, shadows and highlights the tone equalizer, and even the tone curve and the local contrast (to some extent, when used with blend mode multiply)
2. The channel mixer module can overcome all your gamut problems, including problems with blue in stage lighting, without having to use a fake input profile, but also turn grass into snow or summer trees into fall trees
3. The color balance module can allow you to emulate the colors of a film, compensate for uneven white balance, remove redness on the skin, accentuate the depth and shape, create a split-toning effect, or to give an apocalyptic atmosphere to your images

Finally, to display only a minimal selection in the interface and modules, to the right of “More modules“, open the list of presets and select “workspace: all-purpose“.

darktable is a lot simpler when you understand that you don’t have to use all 77 of its modules at once …

If you have any doubts about the order of the modules, you should know that the default order for version 3.0 has been considered globally, and, apart from some uncertainties on the best position of the vignetting and monochrome modules, the rest is pretty solid, in theory and practice.

Conclusion

Pushing pixel values in either direction is one thing. Merging the corrections so they blend seamlessly together on the whole is another. We’ve seen that Lab or non-linear RGB allow the pixels to be pushed more or less correctly, but that it is always when doing mask blending (aka occlusion) and feathering (aka blurs) that we’re paying the price. It turns out there are a lot of blurs under the hood of darktable, sometimes where you don’t expect them. It’s especially problematic when you’re compositing, e.g. inlaying one image within another, to exchange their background without touching the foreground. And it’s precisely this kind of manipulation that led the movie industry to migrate to a scene-referred linear workflow about twenty years ago.

So darktable is in transition. It’s long, it’s sometimes painful, there are a lot of little bits to change in different places along with grumbling users who are hungry for consistency. At least now you know the why and the how. You also know what you have to win. I hope this helps you move forward.

For new users, limit yourself to the above recommended modules, and venture further when you begin to be comfortable. For older users, the new modules have a lot to offer to you, but old Lab modules are still relevant for moderate creative effects and when used with knowledge of their dangers.

The linear toolbox is being expanded. On the agenda:

• rewriting the 100% RGB color balance (including the blending), with the addition of vibrance (and a vibrance equation home-developed to preserve the color)
• conversion of the contrast equalizer and soften modules to the linear xyY space (because in fact, the Orton effect, on which the soften module is based, is very useful when it works correctly)
• a color equalizer, similar to the tone equalizer, which will allow you to adjust saturation, vibrance and Abney effect according to the pixel luminance, to pep up the filmic RGB curve
• a brand-new lens deconvolution module, respectful of the depth of field (but for that, I need to develop a special wavelet based on the guided filter), which should turn your soft 18-55 mm into a Zeiss for much less
• and of course the OpenCL version of the tone equalizer

There is more work than people to do it, so wish us good luck, don’t forget to support us, and Happy New Year 2020 to all of you!

Portrait photographer in Nancy-Metz. Calculation specialist, modeling and numerical simulation for image processing (denoising, deblurring, colour management) and thermal engineering. Developer of filmic RGB, tone equalizer, color balance, and the new themeable interface for darktable 3.0. darktable user since 2010. darktable is my job, so help me out to develop.

G'MIC 2.7 - Process Your Images with Style!

The IMAGE team at the GREYC research laboratory is pleased to announce the release of version 2.7 of G’MIC (GREYC’s Magic for Image Computing), its free, generic, extensible, and probably a little magical, framework for digital image processing.

The previous PIXLS.US article on this open-source framework was published a year ago, in August 2018. This new release is therefore a good opportunity to summarize the main features and milestones of the project’s life over the past twelve months. Fasten your seat belts, the road is long and full of surprises!

---

Useful links:

• The G’MIC Project
• G’MIC Twitter Feed
• G’MIC Forum on PIXLS.US

---

1. G’MIC in 300 words

G’MIC is a piece of software that has been developed for more than 10 years now, mainly in C++, by two members of the IMAGE team of the GREYC lab: Sébastien Fourey and David Tschumperlé. It is distributed under the terms of the CeCILL free-software license. GREYC is a French public research laboratory located in Caen, specialized in digital sciences, under the head of three academic institutions: CNRS, University of Caen, and ENSICAEN.

The IMAGE team, one of the seven teams in the laboratory, is composed of researchers, professors, Ph.D. students and engineers, all specialized in the fields of algorithmics and mathematics of image processing.

G’MIC is cross-platform (GNU/Linux, MacOS, Windows, …). It provides various user interfaces for manipulating generic image data, i.e. 2D or 3D hyperspectral images or sequences of images with floating-point values (which indeed includes “usual” color images). Around a thousand different processing functions are already available. However, arbitrarily many features can be added thanks to an integrated scripting language.

The most commonly used G’MIC interfaces are: the gmic command, that can be accessed from the command line (which is an essential complement to ImageMagick or GraphicsMagick), the G’MIC Online Web service, but above all, the plug-in G’MIC-Qt, available for the well-known image editing software GIMP, Krita, and Paint.net. It provides more than 500 different filters to apply on images.

Thanks to its extensible architecture, G’MIC is regularly enhanced with new image processing algorithms, and it is these latest additions that will be discussed in the following sections.

2. Add style to your images!

G’MIC has recently implemented a neat filter for style transfer between two images, available from the G’MIC-Qt plug-in under the “Artistic / Stylize“ entry. The concept of style transfer is quite simple: we try to transform an image (typically a photograph) by transferring the style of another image to it (for example a painting).

The implementation of such a style transfer method is relatively complex: The algorithm must be able to recompose the original photograph by “borrowing” pixels from the style image and intelligently combining them, like a puzzle to be reconstructed, to best match the content of the data to be reproduced, in terms of contours, colors and textures. How easily this is done depends of course on the compatibility between the input image and the chosen style. In computer graphics, most existing implementations of style transfer methods are based on convolutional neural networks, more particularly generative adversarial networks (GANs).

G’MIC implements style transfer in a different way (without relying on neural networks, the scientific article detailing the algorithm is currently being written!). This method is parallelizable and can therefore benefit from all the processing units (cores) available on the user’s computer. The computation time naturally depends on the input image resolution, and the accuracy of the desired reconstruction. On a standard 4-cores PC, it could take tens of seconds for low resolution images (e. g. 800x800), up to several minutes for larger pictures.

As one can imagine, it is a very versatile filter, since we can apply any style to any input image without hard constraints. Some famous paintings are available by default in the filter, in order to propose predefined styles to the user.

Let us be honest, it is not always easy to obtain satisfactory results from the first draft. It is generally necessary to choose your starting images carefully, and to play with the many parameters available to refine the type of rendering generated by the algorithm. Nevertheless, the filter is sometimes able to generate quite interesting outcomes, such as those shown below (the original photo is visible at the top left, the style chosen at the top right, and the result of the style transfer at the bottom). Imagine how long it would take for a graphic designer to make these transformations “by hand”!

Other examples of image stylization can be found on the image gallery, dedicated to this filter. To our knowledge, G’MIC is the only “mainstream” image processing software currently offering a generic style transfer filter, where any style image can be chosen.

A last funny experiment: get a picture of an Alien’s head, like Roswell, and then select a crop of the Mandelbrot fractal set as your style image. Use the transfer filter to generate a “fractal” rendering of your alien head. Then, make the whole world believe that the Mandelbrot set contains the mathematical proof of the existence of aliens… ☺

In short, this filter has a clear creative potential for all kind of artists!

3. Interactive deformation and morphing

This year, G’MIC got an implementation of the RBFs interpolation method (Radial Basis Functions), which is able to estimate a dense interpolated function in any dimension, from a known set of scattered samples (not necessarily located on a regular grid). Thus, it gave us the idea to add distortion filters where the user interaction is focused in adding and moving keypoints over the image. In a second stage, G’MIC interpolates the data represented by these keypoints in order to perform the distortion on the entire image.

Let us start with the “Deformations / Warp [interactive]” filter which, as its name suggests, allows the user to distort an image locally by creating/moving keypoints.

The animation below shows this interactive filter in use, and illustrates the fact that these keypoints can be considered as anchors to the image, when they are moved.

(For those who might be concerned about the portraits photos used in the figures above and below: all these portraits are totally artificial, randomly generated by GANs via the website This Person Does Not Not Exist. No moral prejudices to dread!).

The great advantage of using RBFs-based interpolation is that we do not have to explicitly manage a spatial structure between the keypoints, for instance by defining a mesh (i.e. a “deformation grid”). This gives a greater degree of freedom in the obtained distortion (see Fig.3.3. below). And at the same time, we keep a rather fine control on the local amplitude of the applied distortion, since adding more “identity” keypoints around a region naturally limits the distortion amplitude inside this region.

A short demonstration of this distortion filter is also visible in this Youtube video.

And why not extending this kind of distortion for two images, instead of a single one? This is precisely what the new filter “Deformations / Morph [interactive]” does. It is able to render a morphing sequence between two images (put on two separate layers), using the same interpolation technique that only asks for the user to set colored keypoints which match on both images.

In the example above, keypoints are placed on characteristic areas of both faces (tip of nose, lips, eyebrows, etc.). In practice, this takes no more than 5 minutes. Thanks to these keypoints, the algorithm is able to estimate a global deformation map from one image to another, and can generate temporally “mixed” frames where the elements of the face remain relatively well aligned during the whole morphing sequence.

By comparison, here is what we would obtain by simply averaging the two input images together, i.e. without correcting the displacement of the facial features between both images. Not a pretty sight indeed!

Thus, the morphing filter is able to quickly generate a set of intermediate frames, ranging from the “Source” to the “Target” faces, a sequence that can then be saved as an animation.

Many other use cases of this morphing filter can be considered. The following example illustrates its application to render an animation from two photographs of the same object (a garden gnome), but shot with different DOFs (Depth of Field).

Command line users will be pleased to know that these two filters can be tested very quickly from a shell, as follows:

$ gmic image.jpg x_warp
$ gmic source.jpg target.jpg x_morph
~

4. Ever more colorimetric transformations

For several years, G’MIC has contained colorimetric transformation filters able to simulate the film development process, or to give particular colorimetric moods to images (sunlight, rain, fog, morning, afternoon, evening, night, etc.). In a previous report, we already mentioned these filters, which are essentially based on the use of 3D CLUTs (Color Lookup Tables) for modeling the color transformation.

A 3D CLUT is technically a three-dimensional array that provides for each possible RGB color, a replacement color to apply to the image.

The main interest of these 3D CLUTs is the great variety of transformations they can represent: They can indeed define RGB-to-RGB functions with almost any kind of variations. The only “constraint” of these methods is that all image pixels having the same color will be transformed into pixels that also have an identical color.

The disadvantage, however, is that these 3D CLUTs are relatively data intensive. When you want to embed several hundred different ones in the same piece of software (which is the case in G’MIC), you quickly find yourself with a large volume of data to install and manage. For instance, our friends at RawTherapee offer on their website an additional pack of 294 CLUTs functions to download. All these CLUTs are stored as .png files in a .zip archive with a total size of 402 MB. Even if downloading and storing a few hundred _MB_ is no longer limiting nowadays, it is still quite large for things as simple as color changing filters.

This year, we have therefore carried out important research and development work at the GREYC lab on this topic. The result: a new lossy compression algorithm (with visually imperceptible compression losses) that can generate binary representations of CLUTs with an average compression rate of more than 99%, relative to the data already loslessy compressed. The general idea is to determine an optimal set of color keypoints from which the CLUT can be reconstructed (decompression), and this, with a minimal reconstruction error.

As a result, this original compression method allowed us to offer no less than 763 CLUTs in G’MIC, all stored in a binary file that weights less than 3 MB !

All these color variation filters have been grouped into two separate entries in the G’MIC-Qt plug-in, namely “Colors / Simulate Film” (for analog film simulations), and “Colors / Color Presets” (for other color transformations). Each of these filters provides sub-categories for a structured access to the hundreds of CLUTs available. To our knowledge, this makes G’MIC one of the image processing software with the most colorimetric transformations, while keeping a reasonable size.

Readers interested in the mathematical details of these CLUT compression/decompression algorithms may refer to the scientific paper we wrote about it, as well as the presentation slides that have been presented at the conferences GRETSI’2019 (French conference, in Lille) and CAIP’2019 (International conference, in Salerno).

To finish with this topic, note that we have made an open-source implementation of our decompression algorithm of CLUTs available online (in C++, with 716 CLUTs already included). Discussions have also been initiated for a potential integration as a Darktable module for managing 3D CLUTs.

5. Create palettes by mixing colors

Let us now talk about the recent “Colors / Colorful Blobs” filter which is directly inspired by the original concept of Playful Palette created by the Adobe Research team in 2017. This filter is intended for illustrators (designers and digital painters). The goal: Create color palettes which contain only a few main colors (the ones you want to use in an illustration), but also a few sets of intermediate shades between these colors, in the form of color gradients. An artist is theoretically able to better preserve the color coherence of its artwork, by picking colors only from this palette.

As shown on the figure above, the filter allows the artist to create and move colored “blobs” that, when merged together, create the desired color gradients. The result of the filter is thus an image that the artist can use afterward as a custom 2D color palette.

From a technical point of view, this filter is based on 2D metaballs to model the color blobs. Up to twelve separate blobs can be added and different color spaces can be chosen for the calculation of the color gradient (sRGB, Linear RGB or Lab). The filter also benefits from the recent development of the G’MIC-Qt plug-in that enhances the user interactivity inside the preview widget (a feature we mentioned in a previous report), as seen in the animation below (see also this longer video).

This filter may not be useful for most G’MIC users. But you have to admit, it’s pretty fun, isn’t it?

6. Some more filters

Let us now describe a selection of a few other filters and effects added during the year, perhaps less original than the previous ones (but not completely useless anyway!).

• First of all, the “Rendering / Symmetric 2D Shape” filter is a great help when you want to draw geometric shapes having angular symmetries.

  

  The plane can be subdivided into up to 32 angular pieces, each of which can contain a maximum of six keypoints to define a shape profile, allowing potentially complex and varied shapes to be rendered (such as the super-shuriken below!).

  

• The “Degradations / Self Glitching” filter combines an image with a shifted version of itself, to create a Glitch-art type image. Several bitwise operations (Add, Mul, And, _Or_, Xor,…) can be chosen and you can adjust the shift direction and amplitude, as well as various other controls.

  

  Again, this is not a filter that will necessarily be used every day! But it may be helpful for some people. It was actually added in response to a user request.

• In the same style, the “Degradations / Mess With Bits” filter applies some arithmetic operations to the pixel values, seen as binary numbers (for instance, bit shift and bit inversion). Always with the idea of rendering Glitch art, of course!

  

• The “Degradations / Noise [Perlin]” filter implements the generation of the Perlin noise, a very classical noise model in image synthesis, used for the generation of elevation maps for virtual terrains. Here we propose a multi-scale version of the original algorithm, with up to four simultaneous variation scales.

  

• The “Frames / Frame [Mirror]” filter is also a “tailor-made” effect, to meet the needs of a G’MIC-Qt plug-in user. This photographer wanted to resize his photos to obtain a precise width/height ratio, but without having to crop his images. The solution was instead to add image information at the edges of the picture, by symmetry, in order to obtain the desired ratio. So that’s what this filter does.

  

• Finally, let us mention the upcoming advanced image noise reduction filter, by Iain Fergusson, whose development is still in progress. Iain has been contributing to G’MIC for several years now by implementing and experimenting original denoising filters, and his latest project seems really interesting, with promising results. This video shows this filter in action, a good place to learn a little more about how it works.

Now that we’ve looked at these new filters, it seems important for us to remind that, as in many IT projects, this visible part of the iceberg hides a set of lower-level developments done to improve the interactive possibilities of the G’MIC-Qt plug-in, as well as the performance of the internal scripting language interpreter (the G’MIC language), which is how all these filters and effects are actually implemented. These improvements and incremental slight optimizations of the code base benefit to all filters (even those already available for several years) and it actually represents most of the development time we spend on G’MIC. So, dear users, do not be surprised if no new filters appear for a while. It is probably just because we are doing serious work on the G’MIC framework core!

7. Other notable points in the project life

Here are listed some other important news that have punctuated the life of the project since August 2018.

7.1. We now accept donations!

This is essential news for us: since March 2019, the G’MIC project has been granted permission to collect donations (via Paypal), to help in its maintenance and development!

This is a good thing, because until now, there was no simple way for a public research laboratory as the GREYC, to accept donations for supporting the development of a free software application such as G’MIC, an application used daily by several thousand people around the world. And we have currently no other ways to finance this piece of software in the long term.

Thus, we have partnered with LILA (Libre comme l’Art), a French non-profit organization promoting Arts, Artists and Free Software, who accepted to collect donations for us.

In practice, this is something that has been a little long to set up, but now that the donation system is operational, we hope to benefit from it in the future to make the project development even faster (the possible use of the raised funds is detailed on the donations page, this being of course very dependent on the amount of money collected).

For the sake of transparency, we will post the monthly amount of collected donations on the project website. At this point, we don’t really know what to expect in practice. We will see how these donations evolve. Of course, we would like to thank all those who have already participated (or plan to do so) in supporting our open-source framework for image processing. Our ultimate dream would be, one day, to say that the illustration below is only a distant memory!

7.2. Integrating “Smart Coloring” into GIMP

Let us also mention the work of Jehan, known to PIXLS.US readers as a regular GIMP developer. Jehan has been hired by the GREYC laboratory in September 2018, to work on G’MIC (for a 12-month fixed-term contract), thanks to a grant funded by the INS2I Institute of the CNRS (for which we are grateful).

One of its first missions was to re-implement the G’MIC “Smart Coloring” algorithm (that we had already talked about previously) as a new interactive mode integrated into the existing GIMP “Bucket Fill“ tool.

Jehan described all his work in a blog post, which is strongly recommended for reading. Of course, we don’t want to copy his post here, but we want to mention this activity, and to consider it as another original contribution of the G’MIC project to free software for graphic creation: at the GREYC laboratory, we are really happy and proud to have imagined and developed an image colorization algorithm, which artists can use through a well integrated tool into such a popular piece of software as GIMP!

This intelligent colorization algorithm has been the subject of scientific publications, presentations at the conferences GRETSI’2017, EuroGraphics VMV’2018, as well as at the Libre Graphics Meeting’2019. And it is with a great pleasure we see this algorithm is used in real life, for various realizations (as in this great video of GDQuest, for colorizing sprites for video games, for instance).

Scientific research carried out in a public laboratory, which becomes available for the general public, that is what we want to see!

7.3. Other news related to the G’MIC project

• Recently, a major improvement in the performances of G’MIC under Windows has been achieved, by recoding the random number generator (now reentrant)) and removing some slow mutex which were responsible of performance drops for all filters requiring sequences of random numbers (and there were many!). As a result, some filters are accelerated by a factor of four to six under Windows!

• Since December 2018, our G’MIC-Qt plug-in is available for Paint.net, a free graphic editing software application under Windows (not open-source though). This has been possible thanks to the work of Nicholas Hayes who wrote the glue code allowing the interaction between our G’MIC-Qt plug-in and the host software. Users of Paint.net are now able to benefit from the 500+ filters offered by G’MIC. This plug-in, available here, has already been voted “Best Plug-in of the Year 2018“ by the members of the Paint.net forum ☺ !

• Since October 2018, the G’MIC-Qt plug-in for GIMP has been compiled and proposed for MacOS by a new maintainer, Andrea Ferrero, who is also the main developer of the free software application Photoflow, a non-destructive image editor (more information here). Many thanks Andrea, for this wonderful contribution!

  • Since the announced shutdown of the Google+ social network, we have opened two new accounts, on Framasphere and Reddit, to share news about the project’s life (but the Twitter feed is still our most active account).

• Let us also thank Santa Claus, who kindly brang us a materialized version of our mascot “Gmicky” last year. That looks almost perfect!

  

• The G’MIC project was presented at the FENO, the “Fête de l’Excellence Normande“, from 12 to 14 April 2019, at the Caen Exhibition Centre. We were hosted on the stand of the CNRS Normandie, and we carried out demonstrations of style transfer (teaser) and automatic illumination of clip arts (teaser), for the general public.

  

• And to dig even deeper, here are some other external links we found interesting, and which mention G’MIC in one way or another:
  • A video presentation of the plug-in G’MIC-Qt, by Chris’ Tutorial;
  • The Youtube channel MyGimpTutorialChannel offers a lot of videos showing how to use G’MIC-Qt in GIMP to achieve various effects (mostly in German);
  • The Clinic, a Chilean weekly newspaper, apparently used G’MIC to achieve an effect on one of its covers (via the smoothing filter “Artistic / Dream Smoothing”);
  • Another video tutorial, showing how to use the G’MIC “Artistic / Rodilius” filter to create stylized animal photos.

8. The future

As you see, G’MIC is still an active open-source project, and with its 11 years of existence, it can be considered as mature enough to be used “in production” (whether artistic or scientific).

We have never defined and followed a precise roadmap for the project development: the functionalities come according to the needs of the developers and users (and the limited time we can devote to it!). At the moment, there is a lot of interest in image processing methods based on neural networks, and deep learning techniques. It is therefore possible that one day, some of these methods will be integrated into the software (for instance, we already have a prototyped code running in G’MIC that actually learns from image data with convolutional neural networks, but we are still at the prototyping stage…).

After 11 years of development (make it 20 years, if we include the development of the CImg library on which G’MIC is based), we have reached a point where the core of the project is, technically speaking, sufficiently well designed and stable, so as not to have to rewrite it completely in the next years. In addition, the number of features available in G’MIC already covers a large part of the traditional image processing needs.

The evolution of this project may therefore take several paths, depending on the human and material resources that we will be able to devote to it in the future (for the development, but also in project management, communication, etc.). Achieving an increase in these resources will undoubtedly be one of the major challenges of the coming years, if we want G’MIC to continue its progress (and we already have plenty of ideas for it!). Otherwise, this image processing framework might end up being just maintained in its current (and functional) state. It is of course with a hope for progression that we have recently set up the donation page. We also hope that other opportunities will soon arise to enable us to make this project more visible (you are invited to share this post if you like it!)

That’s it for now, this long post is now over, thank you for holding on until the end, you can resume normal activity! I’ll be happy to answer any questions in the comments.

---

Post-scriptum: Note that the 3D animation displayed as the teaser image for this post has been actually generated by G’MIC, via the command $ gmic x_starfield3d. An opportunity to remind that G’MIC also has its own _3D_ rendering engine capable of displaying simple objects, which is very practical for scientific visualization! We may have the occasion to talk about it again in a future post…

A special thank you for reviewing and helping to translate this article to:
Patrick David, Sébastien Fourey, Christine Porquet, Ryan Webster.

Quick digiKam Tip: Back up digikamrc file

digiKam stores the current state of the application in the ~/.config/digikamrc file. This file keeps track of pretty much everything: from the database connection profile and custom toolbar settings, to the last-used curve and sharpening parameters. So next time you install or reinstall digiKam, don’t forget to back up the digikamrc file. This way, you don’t have to configure a fresh digiKam installation from scratch. Simply copy the file to a safe location or external storage device, and drop the file into the ~/.config folder before you run digiKam.

Location Tracking for Photographers with GPS Logger and Trekarta

When it comes to Android apps for photographers, we are spoiled for choice. From depth-of-field and golden hour calculators to sun position and remote control apps – there are plenty of clever tools to choose from. But there is one particular app combination that can prove to be indispensable for any photographer on the move: a GPS logger and a GPX viewer. There are two main reasons for that.

1. Tracking your movements and saving them in the GPX format can come in handy for geotagging photos.

2. The ability to attach comments to the current location allows you to use the GPS logging app to note places you either photographed or you plan to photograph later. You can then use a GPX viewer app to see and manage bookmarked locations.

There are several apps that offer GPS logging and viewing, but you can’t go wrong with GPS Logger for Android and Trekarta. Both apps are released under an open source license, and they are available free of charge on Google Play and F-Droid.

How you set up GPS Logger for Android is a matter of personal preference. One way to go is to configure the app to automatically start tracking on boot and upload tracks to the desired destination (e.g., a NAS or a file sharing service).

Once GPS Logger for Android is running, adding a comment to the current location is as easy as pulling down the notification drawer and tapping Comment. The app saves the tracks as GPX files in the Android/data/com.mendhak.gpslogger/files directory on your Android device. To view a GPX file in Trekarta, use a file manager to navigate to the directory, and use Android’s sharing functionality to send the desired GPX file to Trekarta.

Processing a nightscape in Siril

A basic tutorial

Siril is a program for processing astronomical photographs.

In this tutorial, I’ll show you how to process a nightscape in Siril 0.9.10.

It doesn’t intend to be comprehensive tutorial but rather to present a basic general workflow that is a good starting point for those who want to learn Siril.

For this purpose, I’m sharing the raw files I used for the image I presented here, except that for this tutorial I limited the number of frames for the sake of bandwidth and processing speed.

You can find and download the raw files here (~1GB).

Setup

The raw files are placed in specific sub-folders according to their use:

• bias/offset frames → ./Bias (20 files)
• dark frames → ./Darks (15 files)
• flats field frames → ./Flats (15 files)
• main subject/light frames → ./Lights (10 files)

Bias, dark, and flat field frames are also called “calibration” frames, their purpose being to improve the quality of the image by correcting the signal-to-noise ratio (in the case of bias and dark frames) and vignetting (with the flat frames). There are several places where you can learn more about the different types of frames for astrophotography.

At the root of the folder, I placed two text files with the .ssf extension, these are scripts used by Siril for batch processing the files. Quite useful. If you want to run a script from Siril, place the .ssf files in ~/.siril/scripts. Upon restarting Siril, a new Scripts menu appears in the top menu bar, allowing you to launch the installed scripts.

I suggest you download the whole folder (~1GB), and move the scripts as indicated above. This way, if you set the working directory in Siril to the root of the folder, launching the script named processing_from_raw.ssf will automagically process the raws and create the output image in both .fit and .tif (16-bit) formats. Please note that in order to successfuly run the scripts, there must be a folder structure like the one used in this tutorial.

Step-by-step processing

I will present the steps I used to process an image of the Milky Way. I don’t know if it’s the best way, but it’s probably close to what the developers of Siril advise to do for the general case of starting from raw files (actually, I started from one of their scripts and just slightly adapted it).

We will start with processing the calibration files, and then processing the lights.

Preparing the bias frames

1. Set the working directory to the Bias sub-folder by clicking on Change dir….

2. We will use the 20 bias frames to generate a master-bias frame. To load the bias frames, click on the + button as shown (make sure that you select RAW DSLR Camera Files in the combo box) and select the bias frames located in the Bias subfolder.

3. In the “Sequence name” field, enter bias (or whatever you see fit) to set the prefix of the sequence and subsequent files, and click Convert to convert the files to the FITS format, which is the main format used by Siril. Note that you don’t need to demosaic the files yet, make sure the Debayer box is unchecked.

When done converting the bias frames, a window will pop up showing a preview of one of the bias frames. Note that since it’s not demosaiced, it will only show as a B&W channel image.

At this point, the bias frames are loaded and ready to be processed to make a master-bias frame.

4. In the Stacking tab, choose Average stacking with rejection as stacking method, and No normalisation under the normalisation combo box. You can leave the Sigma parameters at their default (unless you know or want to experiment for better values).

It should look like this:

5. Click on the Start stacking button. The resulting master-bias frame will be saved as bias_stacked.fit in the Bias subfolder.

Preparing the flat field frames

Since the flats also contain the sensor readout noise (contained in the bias frames), we should remove it by subtracting the master-bias.

1. In the File conversion tab, remove the files already loaded by clicking on the button located just below the - (minus) button, and by clicking on the + (plus) button select and load the flat frames located in the Flats subfolder.

2. Set the working directory to the Flats sub-folder by clicking on “Change dir…” and set the Sequence name as “flats”.

3. Like for the bias frames, ensure Debayer is unchecked, then click on Convert.

4. In the Pre-processing tab, check only the Use offset box, click on Browse to select the Bias/bias_stacked.fit file, and click on Start pre-processing.

5. To generate the master-flat, go to the Stacking tab, and this time set Normalisation to Multiplicative and the Stacking Method as Average with rejection.

6. Click on Start stacking to produce the pp_flat_stacked.fit master-flat frame in the Flats subfolder.

Preparing the dark frames

As with the bias and flats, you need to load the dark frames.

1. In the File conversion tab, remove the files already loaded, select and load the dark frames located in the Darks subfolder.

2. Set the working directory to the Darks sub-folder by clicking on Change dir…, and set Sequence name as darks.

3. Debayer should be unchecked.

4. Click on Convert.

5. The darks need to be stacked the same way as the bias frames. In the Stacking tab, choose Average with rejection and No normalisation.

6. Click Start Stacking.

The master-dark frame is saved as Darks/dark_stacked.fit.

Note: if you take images often in the same conditions (same air temperature, same exposure settings), you can save the dark_stacked and pp_flat_stacked files, and re-use them to process future light frames faster. I read on some forums that some astrophotographers keep their calibration files and use those for around 1 year, before taking new calibration frames.

Preparing the light frames

Now it’s time to start processing the light frames, by first subtracting the darks (which also contain the bias signal) and the flats (from which bias has already been subtracted).

1. Select the light frames in the File conversion tab.

2. Set the Sequence name to lights, and point the working directory to the Lights (Change dir...).

3. Convert the files, still without debayering.

4. Then go to the Pre-Processing tab, check Use dark, select the Darks/dark_stacked.fit file, check Use flat, and select the Flats/pp_flat_stacked.fit file.

5. Make sure that the other boxes are checked as in the following screenshot.

Note that “Cosmetic Correction” can also be done from the “Image Processing” tab.

6. Click Start pre-processing.

This will produce new FITS files with the prefix pp_light_ and the corresponding .seq file. These files are loaded.

Demosaicing the files

It’s time to demosaic our processed files. There’s something strange in the GUI, in that after pre-processing, when you uncheck “Use dark” and “Use flat” boxes, the “Debayer FITS images before saving” and the “Start pre-processing” button become grayed out.

1. In the File conversion tab, remove the selected files and load the 10 pp_light_000xx.fit files.
2. Check the Debayer box and write db_pp_light as the sequence name.
3. Click Convert.

The pre-processed lights will be saved as FITS files, and the corresponding db_pp_light.seq file loaded. Two preview windows will open this time, one with the 3 RGB channels separated, and one with the RGB composite image.

4. In the Register tab, select Global Star Alignment (deep-sky) from the registratrion method drop-down list and click Go register.

If you have more 8GB of RAM, you can try checking the Simplified Drizzle x2 box (it will up-sample the images by a factor 2, increasing the RAM usage by a factor 4). Siril will detect the stars and register each of the 10 images. The preview windows will be updated. By the way, you can play with the zoom and select AutoStretch to get a better preview of the selected image.

5. In the Stacking tab, make sure that Average with rejection is selected as the stacking method, and that Additive with scaling is set for Normalisation.

6. Click on Start stacking.

The resulting aligned and stacked image will be saved as Lights\r_db_pp_light_stacked.fit.

7. At this step, you can also save the resulting image as JPEG, TIFF, PNG, etc. for further processing in your favorite image editor. On the menu, just click on File > Save As, and pick the image format you wish (or right-click on the RGB windows and pick the format that best suits you).

Post-processing the image

Siril can do some more or less specialized post-processing to your image. I found it interesting to use.

• While the stacked image is still loaded in Siril, you can apply a log transform (it is in linear mode in Siril). I haven’t found how to do it in the GUI, but you can simply type “log” in the “Console” field at the bottom of the Output logs tab, in the main window.
• Still in the console field, you can use the command “crop” followed by the coordinates of the bounding box in pixels, to crop the image (some auto-detection tools in Siril require the image to be cropped to remove the borders introduced by aligning the images, in order to work properly). For example, my image can be cropped by typing crop 30 30 5950 3970.
• You can apply green noise removal in the “Image Processing” tab > “Remove Green Noise…”.
• Lucy-Richardson deconvolution can be applied in “Image Processing” menu option > “Deconvolution…”. 10 iterations and a Sigma value of 0.6 are a good starting point.

The resulting image can be saved as JPEG, TIFF, PNG, etc. for further processing in your favorite image editor or as a finished image if you’re satisfied.

Processing for the foreground

The problem with this whole process, is that because the images have been aligned with the stars as reference, the foreground will be blurred because earth moved between successive frames. What I do is to reprocess the light frames from just after the calibration step (i.e. after the dark and flat frames subtraction) but only skipping the stars registration step. By doing so, the foreground will undergo the same pre- and post-processing, and the resulting image will have a sharp foreground and trailing sky.

I provided a script (processing_from_raw_foreground.ssf) which will do that for you, if you already used the first script or if you use the same file naming convention as in the script.

Finally, in your favorite image editor, you can combine the “sky” and “foreground” images using a mask, to get both the sky and the foreground sharp.

Here’s what I obtained following these steps (but using the scripts), after just combining the 2 images in Gimp:

And after quick curve and saturation tweaking in Gimp:

New Topic Previews

Image previews now available for some categories

I’ve been a member of the community over at blenderartists.org (previously elysiun) for a long time (it’ll be 15 years this October according to my profile there). So it was nice to see when they finally transitioned to using Discourse a little while back.

What I really liked, though, was the work that Bart did to specific pages and tag lists to display them. Here’s what their current homepage looks like:

They use a different default main page style, “Categories” view, than we do (“Latest”). This just shows the site categories as a column on the left, then the latest posts in a column on the right.

The row of featured images along the top is actually part of a plugin that I’ll get to in a moment.

If you want to change your own default main page view of the forums, you can modify it at your account Preferences → Interface (and change it to Categories):

Fancy Category Views

The default landing page is neat, but what they did with their forum gallery page is much neater:

They set up the Topic List Previews plugin so the entire category is actually viewed as a tile of images. I think we can all agree that this is generally a much nicer way to view categories that are heavily image-based. Of course, I thought this was a natural fit for us as well!

So through the magic of having an invaluable resource like a darix, he was able to make it a reality for us!

We’ve got it implemented now on the Play Raw category (now its own sub-category under the Processing category), the Critique, and the Showcase category. If you haven’t had a chance to check it out yet, please do. (darix announced it in the thread Play raw posts and you so feel free to give us any further feedback in that topic.)

Keep in mind that for those categories, the preview image will correspond to the first image in the first post. Try to remember to make the first image in those category topics the one you’ll want in the preview.

We haven’t enabled the featured row of images after some initial feedback. We may revisit it again at some point, but hopefully the Play Raw and Showcase categories will look a little better now. It certainly makes the categories a little easier and faster to navigate now that you can see the previews directly on the page.

G'MIC Finally Accepts Donations

Help support an awesome team!

For years the incredible team over at G’MIC (GREYC’s Magic for Image Computing) have been producing an incredible image processing system and many awesome filters to go along with it. They’ve got an very active and awesome community right here on their forums and they’ve been producing all manner of neat processing filters for photographers, digital artists, and scientists.

Due to the project being under the auspices of a French Research Lab, the GREYC laboratory in Caen, France, they were limited in being able to accept any donations.

Until now!

To avoid burying the lede, go and make a donation to the fabulous folks of the G’MIC project: https://libreart.info/en/projects/gmic.

I first heard about GREYCstoration (proto-G’MIC) a long time ago (over 10 years) as the only really viable Free Software image denoising option for photographers. It allowed me to de-noise images on par (or better in many cases) with the then popular Noise Ninja. It’s been an essential part of my toolkit ever since!

Back then David Tschumperlé was really only looking for postcards from users as a “thank you” and maybe some occasional donations to pay for hot chocolates during the day. I finally got to buy him a milkshake while at the Libre Graphics Meeting 2014 in Leipzig, Germany and for the value he has provided me with his software I owe him many, many more!

Value is exactly what I want to bring up in this post. I’m sure many here have had an opportunity to use G’MIC in some form and any attempt at listing the wide range of filters and capabilities it provides would not do it justice. If you’ve realized some value from the project, now is the time to show some love.

I’ve been lucky to call David a friend for a long time now and I can personally attest to his kindness and sincerity. With that in mind I implore you: if you have a few spare dollars, euros, yen, pesos, or gold bullion please consider donating to the project and make sure the milkshakes (or hot chocolates) never stop flowing. David (and Sébastien Fourey!) are a sound investment in providing high value to Free Software artists and photographers!

Donate to the G’MIC project!

(David is too modest to really come out and ask for support but “modest” isn’t really in my vocabulary - so go donate!)

LILA

For a long time David was unable to accept donations from community members. There were some concerns from the research institution his lab is a part of in France. Just recently, though, they managed to reach an agreement where the funds flow through a French non-profit called Libre Comme L’Art, LILA, that includes the fabulous Jehan Pagès of the GIMP team as a member and where the ZeMarmot animated film is being produced.

A big “Thank You!” to Jehan and LILA for making this possible!

A Q&A with the CHDK Developers

Adding Features to a Camera: Hacker Meets Photographer

Introduction

CHDK is a free, open source software add-on that runs on Canon PowerShot cameras and expands their functionality. Some of its features are:

• Professional control: RAW files, bracketing, manual control over exposure, zebra mode, live histogram, grids, etc.
• Motion detection: Trigger exposure in response to motion, fast enough to catch lightning.
• USB remote: Simple DIY remote allows you to control your camera remotely.
• Scripting: Control CHDK and camera features using uBASIC and Lua scripts. Enables time lapse, motion detection, advanced bracketing, and more.
• PTP: Shooting control, live view, and file transfer from Linux and Windows.

I talked with the core team of developers to learn more about CHDK.

How did CHDK start? Who were the first developers? What was their role in those first steps? Do you have any information on who those people are, where they come from, or their professional background?

reyalp: That was before my time. The very first developer was VitalyB, and I don’t know much about his background.

In truth, CHDK is very loosely organized and informal, so I don’t know much about current contributors backgrounds either.

waterwingz: Before my time too. The original hack seems to have taken off in 2006/2007 when it got mention on the dpreview.com site. Many people shared random bits and pieces based on what they were personally interested in working on. There was no project organization and people did their own builds.

At some point, I believe someone with the nick GrAnd, got things organized around a wikia site and created a standardized set of build tools. Eventually an online discussion forum and autobuild server were added - not sure who gets credit for those.

But over the last 10 years, more or less, reyalp has coordinated the ongoing volunteer development efforts. There is still no real plan or schedule but there seems to be community consensus of how things get done and what gets added to the package.

Some people say that CHDK was first developed by Andrei Gratchev (here and here). I believe he is GrAnd, right? Do you know something about it? Is it possible that Andrei Gratchev is VitalyB?

reyalp: GrAnd (Andrei Gratchev) and VitalyB are definitely not the same person. They have separate accounts on assembla.com. VitalyB did the very first work on what eventually became CHDK, while GrAnd was an early developer who played a major role in organizing the project. It’s possible that GrAnd originated the name CHDK. I don’t know that history directly, but it could explain confusion over whether he was the founder.

Canon cameras run on DIGIC boards. As far as I’m aware of, at the time VitalyB did the first hack, people had already hacked DIGIC-I compact Canon cameras and could execute custom programs. What was the big leap (in terms of development or finding hooks in the DIGIC-II firmware) that VitalyB made?

reyalp: The original Digic-I cameras were actually ROM DOS running on a 16 bit x86 clone (except for S1 IS, which is VxWorks on ARM and has a partial CHDK port developed by srsa_4c).

A hack for the DOS based cameras was developed by a Czech developer.

I don’t know if VitalyB was aware of this work, but the platforms were so different there wouldn’t likely have been much overlap.

waterwingz: As far as I know, the big leap in hooking the DIGIC-II firmware was figuring out how to hack Canon’s firmware update process.

It’s stated that he hacked the firmware update process and executed his own program instead of the firmware update itself. That first program aimed to make a copy of Canon firmware. How exactly could he get a copy of the Canon firmware by blinking a LED? Why did he need a copy of Canon firmware?

waterwingz: From what I understand, he assembled a little piece of code that loaded and ran in the place of the expected firmware update code.

Once he could do that, by trial & error he learned what memory address needed to be poked to turn one of the camera’s LEDs on & off. And once he could do that, he recoded so as to dump the camera’s memory contents serially via that LED to a phototransistor interfaced to an external computer.

After that it was a matter of reverse assembly of the raw code to learn how the rest of the boot process and camera firmware worked.

reyalp: I don’t know the specifics of exactly what VitalyB did for the very first camera, but generally to make a hack work with the existing firmware, you need a copy of the firmware code to disassemble and analyze.

The advantage of using LED blinking is that the code is really simple: you just need to know how to control an LED (done writing to specific address on these cameras) and a loop.

In contrast, writing a file to the SD card requires a whole stack with an SD driver, a filesystem driver and so on.

Without having already analyzed the firmware, you don’t know how to interface with those things, and on PowerShot cameras, they aren’t really available after a firmware update file is loaded.

Blinking was used frequently in the early days of CHDK, but around 2010 Alfredo Ortega and Oren Isacson of Core Labs worked out how to run scripts in Canon’s native scripting language (which we call Canon Basic). I wrote a script to dump the original firmware from Canon Basic, and we’ve used that as the primary way of dumping firmware ever since.

Canon DSLR’s also allow this same kind of hack. Why did VitalyB start with point-and-shoot cameras?

waterwingz: I’m guessing, but probably he started there because he happened to own a PowerShot and not a DSLR. Or maybe it was just the first device that he was able to find a backdoor into.

reyalp: While Canon DSLRs and P&S use the same basic CPU and operating systems, the rest of the code is very different.

Running custom code on Canon DSLRs uses different mechanisms which weren’t figured out until much later.

My impression is VitalyB started on the camera he had (PowerShot A610?).

The lower cost of P&S also makes them more attractive to experiment with, and before the rise of smartphones, P&S were much more common than DSLRs so there was a better chance of interested developers having them.

Are there any ties between CHDK and Magic Lantern (a CHDK equivalent for Canon DSLRs)?

waterwingz: There are several people who participate in both projects and some discoveries are occasionally useful to both. But there is no coordination beyond that.

reyalp: We do share information and occasionally code, but as mentioned above, the Canon firmware up to now has been quite different. Some of the initial DSLR research took place on the CHDK forum.

CHDK was born as HDK, or Hack Development Kit, and only later the “C” was added. Regarding the idea behind the name, what exactly does it mean to say that CHDK is not a simple firmware add-on, but a development kit? Does it have to do with the capability of loading and executing custom user scripts?

waterwingz: Way before my time again - I’d only be speculating on the name’s origin. I guess the kit designation means you get the source code to do development with. I don’t think it has anything to do with user scripting capabilities.

Is this kind of hack only possible on Canon cameras? Why?

waterwingz: The exact details and mechanism will only work on Canon PowerShot cameras. They all build on how Canon supports firmware upgrades (even though CHDK does not actually modify any of the camera’s firmware).

To do something similar on a different brand of camera, you’d need to find a way to exploit any firmware update method they might provide. If there is no such mechanism, you’d need to get lucky and find some other vector.

reyalp: As waterwingz says, the specifics only apply to these cameras, but in general, most embedded devices are hackable with enough effort.

Manufacturers put varying levels of effort into preventing it, but the success of CHDK (and later Magic Lantern) involves a lot of stuff that just lined up by chance.

Another important factor in these projects is that reverse engineering is additive: the more you build and understand, the easier it is to keep up with upstream changes in new models. It’s also easier for people to add useful features, which gets more people involved and keeps the whole thing going.

Getting to that critical point on an entirely new system requires a lot of effort and/or luck.

Some of the lucky things that lined up to make CHDK take off were:

• The cameras ran VxWorks on ARM 946E-S, both of which had significant public documentation. Canon later switched to their proprietary DryOS operating system, but by that point CHDK had enough built up knowledge to carry on.
• Canon left a lot of diagnostic stuff in the code and didn’t put a lot of effort into stopping unauthorized code from running.
• There were a lot of PowerShots in circulation and they were affordable, which provided more chances of a developer with the reverse engineering skills having one.
• Canon didn’t make any effort to stop it.

Can authorized warranty repair shops refuse to service cameras because of CHDK? Does CHDK use leave any traces?

waterwingz: So far it has not been a problem as CHDK runs in RAM - it makes no permanent changes to the camera. When you turn the camera off, it disappears and you have to reload it the next time you use it.

And while Canon has made no official statements about CHDK one way or the other, there is an email somewhere from someone in Canon tech service stating that as long as CHDK did not modify the camera in any way, there was no warranty issue.

If you remove the SD card containing CHDK prior to sending your camera for service, there is really no way of anyone knowing you’ve used CHDK.

reyalp: CHDK doesn’t normally leave obvious traces, but if the camera crashes, traces of CHDK can appear in an internal crash log the Canon firmware stores in onboard flash memory.

It’s certainly possible that other traces could be present, the cameras have a lot of sub-components that could store their own diagnostic information.

I don’t recall any cases of anyone reporting having warranty service rejected for this, but that doesn’t mean it couldn’t happen.

Could you give a brief explanation of how CHDK is designed? Do you have any kind of diagram that could illustrate it?

waterwingz: CHDK is a volunteer effort and most volunteers would rather code than document in detail.

But you can learn a lot reading the For Developers section of the CHDK Wikia.

A short description is that CHDK loads by hijacking the camera’s firmware update process and then intercepts some of the camera’s RTOS tasks and replaces them with its own tasks. The CHDK tasks typically replicate the functionality of the original camera tasks but add features and functionality not included in the original Canon code.

In what language is CHDK built?

waterwingz: A combination of ARM assembler and C.

philmoz: I would add Lua to that - a lot of useful features are now in the scripts. waterwingz has built some impressive functionality with his scripts.

I suppose we should also include uBasic as we still include some testing scripts written in it. uBasic is very primitive compared to Lua, so Lua scripts are preferred.

Finally there is also the Canon Basic built into the firmware - we use this to do firmware dumping.

What were some of the difficult issues in making CHDK easier to port to new cameras?

waterwingz: Ummm… who said it was easy?

But seriously, most of CHDK is based on guesses about how the original Canon hardware and firmware works. So much of the coding was done on a “try it and see” basis.

What makes it more difficult is changes to the firmware as Canon changes RTOS’s or generations of DIGIC processors.

philmoz: The Canon firmware remained pretty stable for a while. It continued to evolve but there weren’t any real upheavals until Digic 6. This made it easier to improve the tools, and take some of the guesswork out of ports.

We have a “sig finder“ tool that analyses a firmware dump and tries to find the things needed for a port. When I first started this was pretty primitive and a lot of things had to be done manually. I spent some time improving this tool, and I think that made porting a bit quicker until Digic 6.

waterwingz also created a GUI tool for disassembling the firmware in a way that could be used in a port - I added a scripting language to this to automate the generation of some of the files needed for a port, which I think helped with new ports.

With Digic 6 the architecture changed a lot and things slowed down quite a bit. A lot of reverse engineering on the new cameras has been done by srsa_4c, reyalp, ant, and others so things are getting better again.

reyalp: One of the things that makes it difficult is how many there are. The official CHDK source supports over 150 distinct models (many with multiple firmware versions that each require distinct ports), spanning Canon releases from 2004 through 2015.

Essentially the same CHDK code runs on all of them, so if a new model does something different, you have to figure out how to accommodate it without breaking the existing cameras.

On top of that, the developers don’t own most of the cameras, so testing is difficult.

As Philmoz mentioned, with Digic 6, Canon moved to a ARMv7 architecture processor and a new display system with a TAKUMI GPU, which took a lot of work supporting. srsa_4c did much of the initial reverse engineering work, while I took the concepts from Philmoz‘s “sig finder“ and implemented them in a new tool based on the open source Capstone disassembly library.

nafraf: The scripting language developed by Philmoz helped a lot to port until Digic 5+ cameras. Using code_gen tool it was possible to port new models and improve the existing ports. On release 1.3, for example, code_gen was the key tool to add more than 60s exposure on all cameras.

In which OS do these tools run?

waterwingz: CHDK is built using the gcc compiler so I guess the tools run on anything that supports that compiler - Windows and Linux for sure.

I do all my work under Linux although I have a laptop somewhere that runs the Windows tools.

The autobuild server that rebuilds CHDK after each update and provides current downloads runs under Linux.

And there are quite a few other tools that people have created, some of which are Windows only (or using Wine under Linux) and some of which are Java based and will run on Windows, Linux, or MacOS.

philmoz: I use MacOS for CHDK development.

I also have a Linux VM I use for testing batch builds of the entire set of supported cameras, to make sure big changes don’t break the autobuild server.

reyalp: I use Windows on my primary development system, but all the core CHDK tools and build process have supported Linux for as long as I’ve been involved.

My normal CHDK development environment is MSYS shells and gvim.

I also use Linux in VMs and a Raspberry Pi for some things.

Is there any camera emulator that allows to test core code before loading it into the camera?

waterwingz: There are a couple of clever GUI emulators for testing CHDK uBASIC and Lua scripts.

But I don’t believe anyone has really succeeded in making a QEMU emulator for core code development.

All testing is done on actual cameras.

And to date, I don’t believe anyone in the dev community has bricked a camera which says something about the stability of the process!

philmoz: Magic Lantern uses QEMU to run their code in an emulator.

In theory CHDK could do this as well, but to date no-one has invested the time to create the hardware simulation bits needed.

Debugging CHDK is old school - blinking LED’s, printing messages (if you have the display working), writing log files, and lots of trial and error.

reyalp: Not emulation, but I use chdkptp a lot for interactive testing. Being able to dump bits of memory or call functions interactively from a PC console is very useful.

What’s the easiest way for someone to get involved with CHDK?

waterwingz: Download it and use it on a PowerShot.

Learn what it does and how to run scripts.

Then write some scripts on your own, or modify some existing ones.

Finally, do a port for an unsupported camera - pretty much every CHDK dev started off porting and then got hooked on doing more.

blackhole: The easiest way is to use CHDK for something creative. It’s nice to see when users show results that are the product of using CHDK. I think it’s the biggest reward for developers when they see that their work is well-used.

reyalp: The forum is the best place to get involved with the community.

To get involved with development, it really depends on your interests. If there’s something you want to add, either dive into the code or ask for suggestions on where to start.

Ports of additional cameras are always welcome too, and doing one provides a good overview of how CHDK works.

What are some tasks non-programmers can do to help the project?

waterwingz: There has been a lot of work on the CHDK wiki over the years, but there is still a ton to do.

For example, one CHDK user finds time each month to simply correct spelling and the worst of the grammar mistakes on the more popular pages and the CHDK User Manual.

reyalp: If you do something interesting with CHDK, share it in the forum. A lot of interesting projects start as riffs on something someone else explored years earlier.

Documentation always needs help, but for CHDK, a lot of it really requires careful experimentation or knowledge of the source to do well.

nafraf: If you are a CHDK user and find a bug, or a missing function on the port of your camera, please report the bug to the forum and help to test new versions.

Developers don’t have access to all models, then testing and receiving feedback from the users are necessary to help the project.

How healthy is the production of user scripts? Is it easy for a non-programmer to write a script? In what languages?

waterwingz: The huge improvements in mobile phone cameras have really impacted the market for all but the highest end or largest zoom P&S cameras.

Having said that, a core of serious photographers still work on what interests them and depend on CHDK to help capture their artistic vision.

As far as non programmers and script writing, uBASIC is about as simple as a computer language gets and there are lots of example scripts to study.

Lua provides a much richer programming environment, albeit with a bit of a learning curve.

reyalp: Easy… depends on the user.

Because CHDK is a reverse engineered hack on top of an undocumented system, many behaviors are not well specified or understood.

waterwingz improved things a lot by creating a comprehensive reference of CHDK script functions, but developing non-trivial scripts still requires significant effort and a willingness to experiment.

I’ve been using CHDK for 10+ years, and still find myself greping the CHDK source and making test cases to figure out what functions actually do.

All that said, I don’t think learning to write modest CHDK scripts is particularly harder than starting out with javascript or batch files or that sort of thing.

We should note CHDK uBASIC is based on Adam Dunkels code, not the UBASIC written by Yuji Kida for mathematical computing.

How many developers work on CHDK? How does it work?

waterwingz: There have been hundreds of contributors to CHDK over the years.

Currently there is an active core of two or three people doing original work with low level firmware stuff, a couple of people working more on the user experience, several people generating custom scripts for unique photographic opportunities, and a core of maybe ten CHDK experts not doing much coding these days but continuing to provide support to the community.

What uses are people making of CHDK? Is there any that should be highlighted?

waterwingz: There is a nice list on the main page of the CHDK Wiki - unique things like motion triggering, scripting, RAW/DNG, bracketing, and full manual exposure control.

Originally, there was a lot of interest in just getting RAW files from an inexpensive P&S camera.

More recently the focus has been on making good multi-camera rigs with full centralized control using CHDK’s PTP capability. Everything from book scanners, “bullet time” rigs, to full 3D capture for building small replicas of people. And of course the ongoing interest in time lapse videos and kite and drone photography work.

reyalp: I get a kick out of how many different things show up searching google scholar for “CHDK hack”

Currently, what is the main development effort underway? New functionality? Porting CHDK to new camera models?

waterwingz: A lot of the current development is focused on very detailed features that interest the core developers. Not really anything that will revolutionize the CHDK user experience right away unfortunately.

But there are also potentially some interesting new things in the wings if the devs working on them can ever get them finished.

reyalp: There aren’t really any major features undergoing significant development right now.

There are some ideas and experimental stuff being kicked around, like the GUI concept and some work on capturing raw outside the normal shooting process, but what eventually gets added depends on developers time and interest.

I’m trying to wrap up a few things to release CHDK 1.5 before we start major projects in the official development branch.

So there’s a new CHDK GUI underway, and it seems very interesting and more user oriented. Once finished, how will it be released to the many different camera models that already run CHDK?

waterwingz: Oh oh - you noticed that? Did you also see the comment about the dev working on it not being too good at it getting finished? ;)

For what it’s worth, there has been quite a bit of thought about keeping it generic enough so that will run on all CHDK capable cameras. That’s mostly about screen resolution issues but there will probably be other challenges. Touchscreen cameras like the PowerShot N come to mind.

What is the future of CHDK, considering the evolution of technology? How long do you think point-and-shoot cameras will stay on the market, given the rise of smartphones? Do you think the latter will replace the former?

waterwingz: As I mentioned earlier, smart phone cameras continue to improve and that has impacted the low end PowerShots that CHDK does so much for.

CHDK will continue as an interesting project as long as a people enjoy using it and creating new things with it.

blackhole: The P&S cameras with the large zoom will likely survive the competition of smartphones. Smartphones in this area will not be competitive with cameras for a long time. CHDK is likely to have a future in this area.

philmoz: I think there will also be demand for the higher end P&S cameras with larger sensors, although I don’t think the market will be huge.

The number of people with these cameras interested in CHDK is probably going to be pretty small.

Canon’s EOS-M mirrorless cameras can also run CHDK so there is some interest there.

reyalp: Low end, mass market P&S are clearly on the way out. I agree with blackhole and philmoz that higher end stuff will be around for a while to come, but the possibility of running CHDK on future cameras is always uncertain.

As I mentioned earlier, Canon DSLRs and P&S cameras have been based on different codebases, which are different enough that it doesn’t make sense to run the same hack on both.

CHDK supports the EOS M3 and M10 because they are built on the P&S codebase, while Magic Lantern does not support them.

There are signs Canon is moving to a unified codebase (likely motivated by the same market changes) in Digic 8 cameras, which may preclude CHDK as we currently know it.

However, with millions of CHDK capable P&S in circulation, there will still be potential uses for a long time to come.

Separate from smartphones, the rise of things like Rasbperry Pi, dedicated UAV and action cameras etc. have reduced cases where a hacked P&S is a clear win over other options.

In 2008, if you wanted a programmable, multi-megapixel camera with decent optics your choices were very limited and mostly expensive.

In 2019, you have a lot of options other than CHDK, but at the same time, a lot of these things can work well with CHDK too.

I think the collapse of P&S has also affected the pool of potential CHDK contributors: in 2008, a developer with a casual interest in photography would have a P&S, while today they would be more likely to have a smartphone. Someone who wants to tinker with camera software also has a lot more choices.

Could CHDK benefit from the current boom in single board computers and single board microcontrollers, like Arduino, Raspberry Pi, Esp32, Beaglebone, etc? Could those boards add even more functionality to CHDK? How?

waterwingz: Actually, there are quite a few successful projects out there using those little computers to control one or more Canon PowerShots running CHDK.

Basically anything that will support the necessary USB functionality to implement the PTP protocol and the CHDK extensions to the protocol. Applications like bookscanners, remote timelapse capture, photobooths, and multiple camera 3D image scanning.

Here is an example of Arduino and CHDK usage, with a nice PTP GUI.

reyalp: The webcams on this site are based Raspberry Pi’s using chdkptp to control CHDK cameras (chdkptp is a tool which I maintain that allows controlling CHDK cameras over USB from Linux and Windows).

One of the firsts tests with a full rig (72 cameras) shooting at same time. By nafraf, licensed under CC BY-NC 2.0.

A simple test to show the detail of a segment of the rig, how the cameras were mounted and their response after sending the turn off command using chdkptp. By nafraf, licensed under CC BY-NC 2.0.

Do you developers have time to play with CHDK? What are your preferred use?

waterwingz: There are a ton of features in CHDK and I’ve had fun playing with most of them. But for me, it’s mostly about Lua scripting when I’m actually using CHDK.

blackhole: Unfortunately my real life does not allow me to play with CHDK as much as I want.

My favorite use is when popularizing astronomy among children. It is a priceless experience when you see the glow in their eyes when they see the image of the planet they have taken themselves. For me, this is the highest value of CHDK.

reyalp: I use CHDK raw for general shooting, and scripts to do timelapse.

I’ve used motion detection for lightning and fireworks.

I also do some lo-fi astrophotography using CHDK scripts to take lots of exposures to guide and stack in software.

But what I do most is take test shots of my desk while working on the code ;)

When did you joined CHDK as a developer? Why? What is your background? What is your role?

waterwingz: July 2010 according to the records on the CHDK forum. But it seems like it was only nine years ago.

I got started because it combined two of my hobbies - computers and cameras.

Since then I’ve ported several cameras, contributed some original code, helped fix some bugs, coded many scripts, written a lot of documentation on the wiki, and helped a few newbies on the CHDK forum.

philmoz: I started with the G12 port in November 2010.

I’ve been a software developer for nearly 40 years and like waterwingz, photography is a hobby. CHDK looked like fun and I wanted something to keep me programming - my day job was more management than development.

I was pretty active until 2016, when real-life got in the way.

I now do mobile app development full time, so don’t spend much time on CHDK coding these days.

blackhole: I joined in August 2010.

Prior to that I was just reading the forum as a guest and using CHDK on the old A530 and A590 cameras. At that time I was looking for a better solution for cheap-modified webcams, which were then popular in amateur astronomy. The logical solution was to switch to something cheap with a CCD sensor, so the decision fell on Canon cameras and CHDK.

I became interested in the programming, so I started to collect knowledge on the forum and in the end I made my first port.

reyalp: Around 2008, I happened to get a Canon A540 and google “firmware hack” or something like that just for kicks. I had a background in C and assembly, and a somewhat neglected interest in photography going back to film days, so it seemed like a fun thing to play with.

For me, CHDK development was a nice change of pace, a throwback to the early PC days where if you want to draw something, you write directly to video memory instead of going through a bunch of APIs.

As people came and went from the project I somehow ended up being the chief cat herder.

nafraf: I started in June 2012. My first port was A810. I was using CHDK for a multi camera project and it was difficult to find ports of recent cameras.

With CHDK, the exposure time and ISO values can change in 1/96 EV steps. The first part of this video was made using a standard timer with exposure changes of 1/3 EV steps. The second part was made using script isoinc.lua, no post-processing. By c_joerg, licensed under CC BY-NC 2.0.

Is there anything that you would like to add?

waterwingz: Getting involved in CHDK is a bit of a trap. Once you get in, your free time just disappears. But it can be a lot of fun!

philmoz: The people who have worked on CHDK over the years are an amazingly talented, fun, and helpful group. I have learned a lot from this project and really appreciate the willingness to help, and assistance I’ve received.

reyalp: I’d like to thank all the people who have contributed over the years, and Canon for turning a blind eye to it for so long.

blackhole: CHDK is a very fun and creative project. I invite all photographers and programmers to join us and express their creativity through this project and share their experiences with us. In the end, I would like to thank the entire CHDK community for a pleasant companionship for the last ten years.

nafraf: Thanks to all people involved with this project. I have learned a lot during these years.

This video was created by changing the zoom levels between 24mm and 1200mm (35mm). For each zoom level, 2 images were taken at a time. The camera was a Canon SX50 with 200 zoom levels and CHDK. A special script was used. By c_joerg, licensed under CC BY-NC 2.0.

Thank You CHDK devs!

I want to thank CHDK devs again for taking the time and being patient enough to chat with us, as well as sharing images of their CHDK use!

I also want to thank CHDK users Garry George, Peter Laudanski, Andrew Hazelden, c_joerg, and keoeeit for having kindly shared some images and answered questions about how they shoot them!

Finally, I want to thank Pixls members paperdigits and afre for their invaluable support, without which this interview wouldn’t have been possible.

The CHDK community gathers around https://chdk.setepontos.com/ and all official CHDK documentation can be found on http://chdk.wikia.com/wiki/CHDK.

Libre Graphics Meeting 2019

Let's participate!

It’s that time of year again: Libre Graphics Meeting 2019 is fast approaching!

This year the meeting will be May 29 to June 2 in Saarbrücken, Germany. This is extra exciting because Saarbrücken is centrally located enough that we should have a nice representation from projects and community members. Members of both RawTherapee and darktable live nearby and will be in attendance (along with others from those projects and many others).

Participate!

I’m hoping to have a good representation this year, so first and foremost - please, please consider participating by giving a presentation, leading a workshop, or even a quick lightning talk! The Call for Participation page is here:

https://libregraphicsmeeting.org/2019/call-for-participation/

I will make myself available to help in any way I can. If you want a hand with the presentation, design, graphics or whatever please feel free to ping me (also - remember that we try to archive all of our presentations and material in our Github repo so you can grab any of the assets from there as well)! This is a great opportunity to spread the word about what we’re up to and the many, many awesome projects everyone has created, maintained, and contributed to for Free Software photography.

The deadline for submittal is coming up on January 15^th!
If you think you’d like to present or host a workshop please submit as soon as possible.

Cheers!

I am particularly excited about this meeting because a) I don’t get to attend every year so this is the first time I’m able to make it in a few years, and b) this is a great opportunity to really get a bunch of community members to come together! Saarbrücken is on the high-speed rail network so it’s readily accessible from many places (now you have less excuses to not make it).

I love hanging out with y’all. It’s great to nerd out about photography and catch up. Sometimes it really helps to be able to speak face-to-face and this is the perfect opportunity to also be exposed to all manner of other Free Software projects (or to expose ourselves to others?).

Besides, how else am I going to capture some fun photos of y’all?

What’s Going On

We have a few things planned for sure at the meeting and you’re going to be really sad if you miss them by not coming!

1. PIXLS.US BoF (Birds of a Feather)
   This is a special session set aside for a couple of hours for the entire community to get together and chat about what’s going on, what we’d like to do, and what’s coming up.
2. Photowalk
   I’ve been doing photowalks every year that I’ve been because this is almost the ultimate way for me to spend time with folks (unless we can do a beer drinking photowalk all at the same time…).
3. PIXLS.US Presentation/Update
   I’ll (We’ll? Anyone from the community is more than welcome to help me present on this) present on the community and what we’ve done so far and what we’d like to accomplish moving forward. This is our primary way of reporting out to the wider community who we are and what we’re doing.
4. State of the LGM
   This is a couple of slides that are included at the beginning of the program giving an overview of the state of the entire libre graphics ecosystem.

I really, really, really want to be able to add to this list with other presentations (or lightning talks, etc) that the community will give! I may even submit a couple of presentations to talk not just about the community but maybe about our technical work coordinating the forum and providing services for the projects (as well as invite other related projects to come join us).

We can make this an incredible and memorable meeting with a fantastic opportunity to meet friends in person and have a wonderful time!

Goodbye Google Analytics

A little less tracking for the new year

Over on my personal website I decided to stop using third party trackers and assets to keep from exposing visitors to unintended tracking. Third party assets expose a user to being tracked and analyzed by those third (or fourth, or more) parties and honestly this is something the web could use a little (lot) less of. I loved having stats early on when we started this crazy idea for a community and as I mentioned on my blog post, it’s a Faustian bargain to get stats at the expense of allowing Google to track what all the users of the site are doing. No thanks.

I figure it’s the eve of a new year so why not start it out right and reduce the tracking footprint of the site?

This all started by noticing that some new browser feature strips referer information from requests (thanks @darix) and we were using them to target specific areas of websites that we manage comments for. It came to my attention when I was reading the release announcement for digiKam 6.0.0 beta 3.

While fixing that problem, I found that once we fixed the referer requirement problem I was still seeing issues with Privacy Badger blocking our embed code. On further inspection it boiled down to using Google Analytics on our base domain (pixls.us) and having a cookie set by Google, which then got sent with embed requests from other websites (digiKam and darktable). This triggered the heuristic blocking by Privacy Badger.

Honestly, we derive very little value from the analytics for the price (privacy) we pay to use it. Better to simply remove it.

We still do analytics but we own the stack ourselves (thank you so much andabata!). If you want to block our own analytics the domain is: piwik.pixls.us.

2018 PlayRaw Calendar

Chris creates a new calendar for the community

Last year I got an amazing surprise in the mail. It was an awesome calendar of a handpicked selection of results from the years PlayRaw images.

Chris (@chris) put together another fantastic calendar for this year (while juggling kids, too) and it’s too nice to not have a post about it!

It was a really awesome surprise to recieve my calendar last year - and I wish I would have planned a little better to be able to grab a photo of the calendar hanging in my office (it’s my work desk calendar - it never fails to remind me that there’s more fun things to life than work - also that I need to up my processing game… ).

This year Chris has done it again by assembling a wonderfully curated collection of images and edits from the various Play Raws that were posted this year. I’ve plagiarized his post on the forums to put together this post and get some more publicity for his time and effort!

If you get a moment, please thank Chris for his work putting this together!

You can download the PDF: 2018 Play Raw Calendar

Here are the images he chose for the calendar and the edits he included:

month	image title	photographer	editor	license
0	Monkey Business	jinxos	andrayverysame	CC BY-SA
1	Glaciers, Birds, and Seals at Jökulsárlón/Iceland	BayerSe	McCap	CC BY-NC-SA
2	Shooting Into the Sun	davidvj	Adlatus	CC BY-SA
3	The Rail Bridge, North Queensferry	Brian_Innes	Jean-Marc_Digne	CC BY-SA
4	Sunset sea	Thanatomanic	sls141	CC BY-NC-SA
5	Vulcan stone sunset	asn	kazah7	CC BY-NC-SA
6	Venise la sérénissime	sguyader	Thomas_Do	CC BY-NC-SA
7	Dockland side view at night	gRuGo	CriticalConundrum	CC BY-NC-SA
8	Eating cicchetti with ghosts in Venezia	sguyader	msd	CC BY-NC-SA
9	maritime museum	wiegemalt	yteaot	CC BY-SA
10	Alfred’s Vision	jinxos	msd	CC BY-SA
11	Crescent Moon through silhouetted fern fronds	martin.scharnke	gRuGo	CC BY-NC-SA
12	Everything frozen	asn	McCap	CC BY-NC-SA

A preview (also shamelessly lifted from Chris’s forum post):

These Play Raws are a ton of fun and one of the great aspects of having such a generous community to share the images and allowing everyone to practice and play. I am constantly humbled by the amazing work our community produces and shares with everyone.

Thank you to everyone who shared image and participated in processing (and sharing how you achieved your results)! I have really learned some neat things based on others work and look forward to even more opportunities to play (pun intended).

Fun side note: the Play Raws are actually something that began on the old RawTherapee forums. When they moved their official forums here with us it was one of those awesome things I’m glad they brought over with them (the people were pretty great too… :)).

Giving More Thanks

For an awesome community

It is a yearly tradition for us to post something giving thanks around this holiday. I think it’s because this community has become such a large part of our lives. Also, I think it helps to remind ourselves once in a while of the good things that happen to us. So in that spirit…

Financial Supporters

We are lucky enough (for now) to not have huge costs, but they are costs none-the-less. We have been very fortunate that so many of you have stepped up to help pay those costs.

The Goliath of Givers

For the last several years, Dimitrios Psychogios has graciously covered our server expenses (and then some). On behalf of the community, thank you so much! You keep the servers up and running. Your generosity will cover infrastructure costs for the year and give us room to grow as the community does.

We also have some awesome folks who support us through monthly donations (which are nice because we can plan better if we need to). Together they cover the costs of data storage + transfer in/out of Amazon AWS S3 storage (basically the storage and transfer of all of the attachments and files in the forums). So thank you, you cool friends, you make the cogs turn:

• Jonas Wagner
• elGordo
• Chris
• Christian
• Claes
• Thias
• Stephan Vidi
• ukbanko
• Bill Z
• Damon Hudac
• Luka Stojanovic (a multi-year contributor!)
• Moises Mata
• WoodShop Artisans
• Barrie Minney (He’s a long time monthly contributor!)
• Mica

It is so amazing not to have to worry about finding the capital to support our growing community; we just expand things as necessary. It is super great.

If you’d like to join them in supporting the site financially, check out the support page.

Growth

As of today, we have 3135 users, so we’ve continued to grow at a very good rate! Welcome to all the new users.

As you can see from our discuss stats, we’re approaching 500k page views per month:

And our yearly community health is very positive:

gPhoto

This year we added the gphoto project to our list of supported applications! gPhoto is an awesome library for interfacing with your camera. It is used by darktable and entangle to allow you to shoot with your camera attached to your laptop or other device. We’re thrilled that they’ve joined us on the forums!

Natron

Natron is a compositing application mostly used in 3D/video compositing. The main developer was looking to give the project more of a community focus, so of course we were happy to provide them their own spot in the forum for their users to communicate and collaborate.

darix

For another year, @darix continues to keep our stuff up and running! Do you ever notice outages? No?! Me either, and that is due to his daily diligence. We can’t thank him enough for his dedication to our community.

patdavid or Pat David

The originator of it all, thank you for the initial push to create this community where we are not divided by which application we use. And for your continued good will towards everyone here, your welcoming spirit, and passion. We’d never have done it without you! And for all the great things to come!

All of You

The community is the sum of parts + all the extra love that comes from all of you! Thank you so much continuing to stick around, share you knowledge, and spread the great community. It keeps me motivated, creative, and challenged and for that I am very thankful.

Create lens calibration data for lensfun

Adding support for your lens

[Article updated on: 2019-12-09]

Introduction

All photographic lenses have several types of errors. Three of them can be corrected by software almost losslessly: distortion, transverse chromatic aberration (TCA), and vignetting. The Lensfun library provides code to do these corrections. Lensfun is not used by the photographer directly. Instead, it is used by a photo raw development software such as darktable or RawTherapee. For example, if you import a RAW into darktable, darktable detects the lens model, focal length, aperture and focal distance used for the picture, and it then calls Lensfun to automatically correct the photograph.

Lensfun uses a database to know all the parameters needed to do the lens corrections. This database is filled by photographers like you, who took time to calibrate their lenses and to submit their findings back to the Lensfun project. If you’re lucky, your lens models are already included. If not, please use this tutorial to do the calibration and contribute your results.

Let us assume your lens isn’t covered by Lensfun yet, or the corrections are either not good enough or incomplete. The following sections will explain how to take pictures for calibration. It will also show you how to create an entry of your own. The best is to provide information for all three errors but maybe you only need distortion then this is fine too.

Checking if your lens is already supported

Before you start to calibrate new lenses or report missing cameras please check the lens database first! The list is updated daily. If your lens is already support then everything is fine and you just have to update your database.

If the lens is not supported or doesn’t provide all corrections you could add the missing data following this tutorial.

Taking pictures

Before we start you need to take a lot of images for the three errors we are able to correct. This section will explain how to take them and what you need to pay attention to.

For all pictures you should use a tripod, turn off all image correction and disable image stabilization in the camera and in the lens itself! Also make sure to that all High Dynamic Range (HDR) or Dynamic Range Optimizer (DRO) features are turned off. All those options could mess up your calibration.

Distortion

For distortion you can to take pictures of a building with several parallel straight lines. You need at least two lines, one should be at the top of the image (Nearly touching the top of frame) and the other line at about a third down from the first line. The following example demonstrates this.

The lines must be perfectly straight and aligned. You can twist and rotate the camera, but the lines must have no imperfections. Common mistakes are using tiles or bricks: to your eye they may be “straight”, but it will cause calibration defects. The best buildings turn out to be parking houses (US: parking lot, EN: garages, car parks) or modern glass buildings like fruit-technology stores.

For a fixed focal length lens, you only will require one image. For a zoom lens it is recommended to take 5 to N pictures where N is max focal length minus min focal length. You must take an image at the minimum focal length, and the maximum focal length. You can move (step backward on forward) between shots to keep the 1/3rd rule above consistent.

You should shoot at your lenses sharpest aperture - this is often f/8 to f/11. Setup your camera on a tripod. Shoot at the lowest ISO (without extended values). This will be 100 or 200. Disable any inbody lens corrections. Every vendor has a different name for this process (Fuji is modular lens optimization for example). Check your camera manual and menus.

Chromatic aberrations (TCA)

For TCA images look for a large object with sharp high-contrast edges throughout the image. Preferably, the edges should be black–white but anything close to that is sufficient. Make sure that you have hard edges from the center throughout to one of the edges. The best buildings, for taking photos, have dark windows with white or gray frames.

Here are some example pictures:

You should take your pictures being at least 8 meters away. For zoom lenses, take pictures at the same focal lengths as for distortion (5 to N). Make sure to capture really sharp photos using at least f/8. The best is to use aperture control, f/8 and ISO 100 on a tripod to avoid any color noise.

You can use e.g. a streetview service to find the right building in your town (big buildings, dark windows with white or grey frames).

Vignetting

To create pictures for vignetting you need a diffuser in front of the lens. This may be translucent milk glass, or white plastic foil on glass. Whatever, as long as it is opaque enough so that nothing can be seen through it, yet transparent enough so that light can pass through it. It must not be thicker than 3 mm and shouldn’t have a noticeable texture. It must be perfectly flush with the lens front, and it mustn’t be bent. It must be illuminated homogeneously.

I ordered a piece of acryl glass, opal white (milky), smoothly polished, 78% translucency, 3mm thick, 20 x 20 cm, which is about 8 Euro on Amazon.

However white plastic foil taped on a piece of ordinary glass for stability might be enough, if the plastic doesn’t have any texture.

I normally wait for a cloudy day with no sun, then the sky is homogeneously lit. Put the camera on a tripod and point it to the sky. Put the glass directly on the lens (remove any filters). In some places where sunlight is different you may need to shoot indoors. You should experiment to make sure your images are evenly lit (except for vignetting obviously).

Make sure that no corrections are applied by the camera (some models do this even for RAWs). Set the camera to aperture priority and the lowest real ISO (this is normally 100 or 200, don’t use extended ISO values).

Switch to manual focus and focus to infinity. This is the most critical step!

For zoom lenses, you need to take pictures at five different focal lengths. You only need pictures for five focal lengths because for the other steps it gets interpolated. For a prime lens you need to take only pictures for the single focal length.

Take the pictures as RAW at the fastest aperture (e.g. f/2.8) and at three more closed apertures at 1 EV distance, and also at the most closed aperture (e.g. f/22.0). These are often marked on your lens’ aperture ring, or on your electronic display.

If you have for example a 16-35mm lens with aperture f/2.8 - f/22, you need to take pictures at 16mm, 20mm, 24mm, 28mm and 35mm focal length (Remember you require the min and max zoom values). For each for those focal lengths you need to take five pictures at f/2.8, f/4.0, f/5.6, f/8.0 and f/22.0. This makes 25 pictures in total.

For a 50mm prime lens with f/1.4 - f/16 you need to take 5 pictures at f1.4, f/2.0, f/2.8, f/4.0, and f/16.0.

**Exposing the picture correctly

For taking the picture the middle of the area needs to be as bright as possible, but not overexposed. This can be easily +1.7 to +2.0 EV.

If your camera has a zebra setting, turn it on and set the zebra mode to ‘100’. If you start to see the zebra, then take the picure. The profile is created on a RAW file and the zebra is shown for a developed picture. So we aren’t overexposed yet. In the control mode in my camera which shows overexposure everything was fine.

Vignetting correction for the professionals

The following steps are to get really fine grained vignetting corrections. The gain in accuracy is really very small! I probably only makes sense for prime lenses used for portrait or macro photography. However this is not required, the above it absolutely enough.

Lensfun is able to correct vignetting depending on focal distance. Thus, you can achieve a bit more accuracy by shooting at different focal distances. This means you will have to take pictures at 4 different focal distances.

The first focus on the near point (The near point is the closest distance that can be brought in focus). The next focal distances are the near point multiplied by 2 and by 6 and finally focus at infinity.

Example: For a 85mm prime lens with the near point at 0.8 m. You have to take pictures at 0.8 m, 1.6 m, 4.8 m and infinity.

Create calibration data

There are two ways to perform the calibration.

Lensfun allows an upload of data to the project, and they’ll do the program work for you. They’ll also review your images to make sure they are correctly taken.

Or you can do it yourself with the lens calibration script from the lensfun project.

The script needs the following dependencies to be installed on your system:

• python3
• python3-exiv2 (py3exiv2 >= 0.2.1)
• python3-numpy
• python3-scipy
• darktable-cli (darktable >= 2.4.0)
• tca_correct (hugin >= 2018)
• convert (ImageMagick)

You can download the lens calibration script HERE or get it as a package for the major distributions HERE.

Once you have downloaded the tool create a folder for your lens calibration data, change to the directory and run:

$ lens_calibrate.py init
The following directory structure has been created in the local directory

1. distortion - Put RAW file created for distortion in here
2. tca        - Put chromatic abbreviation RAW files in here
3. vignetting - Put RAW files to calculate vignetting in here

Follow the instructions and copy your raw files in the corresponding directories.

Vignetting correction for the professionals

For each focal distance you captures pictures you have to create a folder.

Lets pick up the example from above. For a 85mm prime lens we took pictures at 0.8 m, 1.6 m, 4.8 m and infinity. For this lens you would have to create the following folder structure in the vignetting directory:

vignetting/0.8
vignetting/1.6
vignetting/4.8
vignetting/inf

The folder inf is for the focal distance at infinity.

Distortion

Once you copied the files in place it is time to generate the pictures (tif files) for distortion calculations. You can do this with the ‘distortion’ option:

$ lens_calibrate.py distortion
Running distortion corrections ...
Converting distortion/_7M32376.ARW to distortion/exported/_7M32376.tif ... DONE
A template has been created for distortion corrections as lenses.conf.

Once the tif files has been created, you can start Hugin.

Torsten Bronger created a screen cast to give an overview about the distortion process in Hugin. He uses an old Hugin version in the video. The following section of this tutorial explains how to do it with Hugin 2018. You can watch the screen cast first if you want, you can do it here (Vimeo).

If you start Hugin the first time, the windows you will get should look like in Figure 8.

First select on the menu bar Interface -> Expert to switch to the Expert mode. You will get a windows which should look like as in Figure 9.

Once in the export mode click on Add images (Figure 10) and load the first tiff from the distortion/exported folder.

By default the lens type should be set to Normal (rectiliniar) for normal standard lenses. Make sure that the focal length is correct and set the Focal length multiplier, which is the crop factor of your camera. For full frame bodies this value should be 1. If you have a crop camera you need to set the correct crop value you can find in the specifications. Next click on the Control Points tab (Figure 11).

This is the tab to set the control points so that we can tell the software what are our straight lines we are interested in. In this tab you have to make sure that auto fine-tune is disabled, auto add is enabled and auto-estimate is disabled! If this is the case zoom the image to 200% (you can also do this by pressing ‘2’ on the keyboard).

In the zoomed images you have start at the top edges. On the left go to the top left corner and to the top right corner on the right. The first straight line, from left to right, should be visible. Select the first control point on the left edge of the picture on the left page and the right edge on the right (Figure 12).

IMPORTANT: Once you have the first control point selected in both images. Select Add new Line in the mode dropdown menu! This will add the two control points as line 3! Now continue adding corresponding control points in both pictures till you’re in the middle on both sides.

Tip: The easiest and fasted is to set control points in the middle at the tiling line. This reduces the required mouse movements.

Now zoom out by pressing ‘0’ and check it if everything has been added correctly (Figure 13).

While you are zoomed out, find a line which is about 3rd into the image from the top to repeat adding a line. Zoom to 200% again, select the first control points and again Add a new line which will result in line4 (Figure 14)!

Zoom out by pressing ‘0’ and check that you have two lines, line3 and line4. Now move on to the Stitcher tab (Figure 14).

In the Stitcher tab you need to select the correct Projection for your lens. This is Rectilinear for standard lenses. Once done switch to the Photos tab (Figure 16).

At the bottom under Optimize select Custom parameters for Geometric. This will add an Optimizer tab. Switch to it once it appears (Figure 17).

Select the ‘a’, ‘b’ and ‘c’ lens parameters and click on Optimize now!. Accept the calculation with yes. Now the values for ‘a’, ‘b’ and ‘c’ will change (Figure 18).

The calculated correction values for ‘a’, ‘b’ and ‘c’ you can find in the tab need to be added to the lenses.conf. Open The file and fill out the missing options. Here is an example:

Example:

[FE 85mm F1.4 GM]
maker = Sony
mount = Sony E
cropfactor = 1.0
aspect_ratio = 3:2
type = normal

• maker is should be the lens manufacturer e.g. Sony
• mount is the mount system for the lens, check the lensfun database
• cropfactor is 1.0 for full frame cameras, if you have a crop camera find out the correct crop factor for it.
• aspect_ratio is the aspect ratio for the pictures which is normally 3:2.
• type is the type of the lens, e.g. ‘normal’ for standard rectilinear lenses. Other values are: stereographic, equisolid, stereographic, panoramic or fisheye.

If you have e.g. a 85mm there should be an entry for the focal length which is set to: 0.0, 0.0, 0.0. You need to change the values in the lenses.conf for your focal length with the calculated corrections from the Optimizer tab (Figure 16).

[FE 85mm F1.4 GM]
maker = Sony
mount = Sony E
cropfactor = 1.0
aspect_ratio = 3:2
type = normal
distortion(85mm) = 0.002, 0.001, -0.009

But I don’t want to do distortion corrections!

No problem, if you want to skip this step then you can created the lenses.conf manually. It should look like the following example:

[lens model]
maker =
mount =
cropfactor = 1.0
aspect_ratio = 3:2
type = normal

The section name is the lens model. You can find it out by running:

exiv2 -g LensModel -pt <raw image file>

The other options are:

• maker is should be the lens manufacturer e.g. Sony
• mount is the mount system for the lens, check the lensfun database
• cropfactor is 1.0 for full frame cameras, if you have a crop camera find out the correct crop factor for it.
• aspect_ratio is the aspect ratio for the pictures which is normally 3:2.
• type is the type of the lens, e.g. ‘normal’ for standard rectilinear lenses. Other values are: stereographic, equisolid, stereographic, panoramic or fisheye.

TCA

You can skip this step if you don’t want to do TCA corrections.

This step is fully automatic, all you have to do is to run the following command and wait:

$ lens_calibrate tca
Running TCA corrections for tca/exported/_7M32375.ppm ... DONE

However it possible to calculate more complex TCA corrections. For this you need to run the step it with an additional command line argument, like this:

$ lens_calibrate --complex-tca tca
Running TCA corrections for tca/exported/_7M32375.ppm ... DONE

Vignetting

You can skip this step if you don’t want to do vignetting corrections.

To calculate the vignetting corrections it is also a very simple step. All you have to do is to run the following command and wait:

$ lens_calibrate vignetting

Generating the XML

To get corrections for lensfun you need a lenses.conf with the required options to be filled out (maker, mount, cropfactor, aspect_ratio, type). And at least one of the corrections steps done. If you have this you can generate the XML file which can be consumed by lensfun. You can do it with the following command:

$ lens_calibrate generate_xml
Generating lensfun.xml

You can redo this step as many times as you want. And you can just rerun it if you add an additional correction.

Using the lensfun.xml

You may want to fine-tune the lens model name in the generated lensfun.xml file. Lensfun normalises names before any matching, so you have some freedom. For example, upper/lowercase can be changed arbitrarily. Any single f is ignored, so you may change 16-35mm 2.8 into 16-35mm f/2.8. If there was a tele converter involved, you must add “converter” into the name so that Lensfun does not try to derive allowed focal lengths from the lens name. Ordering of parts in the lens name is completely unimportant for matching. As are single punctuation characters. You may even add things (e.g. 16-35 into 16-35mm) but be conservative here. Never drop anything what exiv2 reports!

If you want to use the generated lensfun.xml file to test if the calibration you created works, you can copy to the local lensfun config folder in your home directory.

cp lensfun.xml ~/.local/share/lensfun

Make sure your camera is recognized by lensfun or you need to add an entry to the lensfun.xml file too.

Contributing your lensfun.xml

To contribute your calibration data data for your lens to the lensfun project, execute the script with the following command:

$ lens_calibrate ship
Created lensfun_calibration.tar.xz
Open a bug at https://github.com/lensfun/lensfun/issues/ with the data.

This will create a tarball with all the required data. Now go to

• https://github.com/lensfun/lensfun/issues/

and open a bug using the following subject:

Calibration data for <lens model>

And for the description just use:

Please add the attached lens data to the lensfun data base.

Attach the lensfun_calibration.tar.xz to the bugreport.

Feedback

Feedback for this article is very welcome. If you’re a lensfun developer and read this please contact me. I would like to contribute the script to lensfun and further improve the article. I still have unanswered questions.

Support Andrea Ferrero on Patreon!

Andrea is developing Photo Flow, GIMP AppImage, Hugin AppImage, and more!

Andrea Ferrero, or as we know him Carmelo_DrRaw, has been contributing to the PIXLS.US community since April of 2015. A self described developer and photography enthusiast, Andrea is the developer of the PhotoFlow image editor, and is producing AppImages for:

• The GIMP image manipulation program - weekly AppImage packages from stable releases and development branches.
• The RawTherapee editor - nightly AppImage and Windows packages from stable and development branches
• LuminanceHDR - nightly AppImage packages for Linux
• HDRMerge - nightly AppImage packages for Linux
• The Hugin panorama photo stitcher - AppImages for stable releases and development branches

Andrea is the best sort of community member, contributing six different projects (including his own)! He is always thoughtful in his responses, does his own support for PhotoFlow, and is kind and giving. He has finally started a Patreon page to support his all of his hard work. Support him now!

He was also kind enough to answer a few questions for us:

PX: When did you get into photography? What’s your favorite subject matter?

AF: I think I was about 15 when I got my first reflex, and I was immediately fascinated by macro-photography. This is still what I like to do the most, together with taking pictures of my kids. ;-) By the way, you can visit my personal free web gallery on GitHub: http://aferrero2707.github.io/photorama/gallery/ (adapted from this project).

It is still a work in progress, but you are welcome to fork it and adapt it to your needs if you find it useful!

PX: What brought you to using and developing Free/Open Source Software?

AF: I started to get interested in programming when I was at the university, in the late 90’s. At that time I quickly realized that the easiest way to write and compile my code was to throw Linux into my hard drive. Things were not as easy as today but I eventually managed to get it running, and the adventure began.

A bit later I started a scientific career (nothing related to image processing or photography, so I won’t bother with more details about my daily job), and since then I have been a user of Linux-based computing clusters for almost 20 years at the time of writing… A large majority of the software tools I use at work are free and open sourced and this definitely has marked my way of thinking and developing.

PX: What are some new/exciting features you develop in Photo Flow?

AF: Currently I am mostly focusing on HDR processing and high-quality Dynamic Range compression - what is also commonly called shadows/highlights compression.

More generally, there is still a lot of work to do on the performances side. The software is already usable and quite stable, but some of the image filters are still a bit too slow for real-time feedback, especially when combined together.

The image exporting module is also currently in a state of work in progress. It is already possible to select either Jpeg or TIFF (8, 16 or floating-point 32 bits bit depth) as the output format, to resize the image and add some post-resize sharpening, and to select the output ICC profile. What is still missing is a real-time preview of the final result, with a possibility to soft-proof the output profile. The same options need to be included in the batch processor as well.

On a longer term, and if there is some interest from the community, I am thinking about porting the code to Android in a simplified form that would be suitable for tablets and the like. The small memory footprint of the program could be an important advantage on such systems.

PX: What other applications would you like to make an AppImage for? Have you explored Snaps or Flatpaks?

AF: I am currently developing and refining AppImage packages for GIMP, RawTherapee, LuminanceHDR and HDRMerge, in addition to PhotoFlow. All packages are automatically built and deployed through Travis CI, for better reproducibility and increased security. Hugin is the next application that I plan to package as an AppImage.

All the AppImage projects are freely available on GitHub. That’s also the best place for any feedback, bug report, or suggestion.

There is an ongoing discussion with the GIMP developers about the possibility to provide the AppImage as an official download.

In addition to the AppImage packages, I am also working with the RawTherapee developers on cross-compiled Windows packages that are also automatically built on Travis CI. The goal is to help them provide up-to-date packages from the main development branches, so that more users can test them and provide feedback.

I’m also open to any suggestions for additional programs that could be packaged as AppImages, so do not hesitate to express your wishes!

Personally I am a big fan of the AppImage idea, mostly because, unlike Snap or Flatpack packages, it is not bound to any specific distribution or run-time environment. The packager has full control over the contents of the AppImage package, pretty much like MacOS bundles.

Moreover, I find the community of developers around the AppImage format very active and open-minded. I am currently collaborating to improve the packaging of GTK applications. For those who are interested in the details, the discussion can be followed here: https://github.com/linuxdeploy/linuxdeploy/issues/2

G'MIC 2.3.6

10 Years of Open Source Image Processing!

The IMAGE team of the GREYC laboratory is happy to celebrate the 10th anniversary of G’MIC with you, an open-source (CeCILL), generic and extensible framework for image processing. GREYC is a public research laboratory on digital technology located in Caen, Normandy/France, under the supervision of 3 research institutions: the CNRS (UMR 6072), the University of Caen Normandy and the ENSICAEN engineering school.

This celebration gives us the perfect opportunity to announce the release of a new version (2.3.6) of this free software and to share with you a summary of the latest notable changes since our last G’MIC report, published on PIXLS.US in February 2018.

---

Related links:

• The G’MIC project
• Twitter feed
• Announcement of the first version of G’MIC on LinuxFr.org [fr]
• Previous article about G’MIC on PIXLS.US

---

(Click on the images of the report to display them in full resolution)

1. Looking back at 10 years of development

G’MIC is a multiplatform framework (GNU/Linux, macOS, Windows…) providing various user interfaces for manipulating generic image data, such as 2D or 3D hyperspectral images or image sequences with float values (thus including “normal” color images). More than 1000 different operators for image processing are included, a number that is extensible at will since users can add their own functions by using the embedded script language.

It was at the end of July 2008 that the first lines of G’MIC code were created (in C++). At that time, I was the main developer involved in CImg, a lightweight open source C++ library for image processing, when I made the following observation:

• The initial goal of CImg, which was to propose a “minimal” library of functions to help C++ developers to develop image processing algorithms, was broadly achieved; most of the algorithms I considered as essential in image processing were integrated. CImg was initially meant to stay lightweight, so I didn’t want to include new algorithms ad vitam æternam, which would be too heavy or too specific, thus betraying the initial concept of the library.
• However, this would only cater to a rather small community of people with both C++ knowledge and image processing knowledge! One of the natural evolutions of the project, creating bindings of CImg to other programming languages, didn’t appeal much to me given the lack of interest I had in writing the code. And these potential bindings still only concerned an audience with some development expertise.

My ideas were starting to take shape: I needed to find a way to provide CImg processing features for non-programmers. Why not attempt to build a tool that could be used on the command line (like the famous convert command from Imagemagick)? A first attempt in June 2008 (inrcast, presented on the French news site LinuxFR), while unsuccessful, allowed me to better understand what would be required for this type of tool to easily process images from the command line.

In particular, it occurred to me that conciseness and coherence of the command syntax were the two most important things to build upon. These were the aspects that required the most effort in research and development (the actual image processing features were already implemented in CImg). In the end, the focus on conciseness and coherence took me much further than originally planned as G’MIC got an interpreter) of its own scripting language, and then a JIT compiler for the evaluation of mathematical expressions and image processing algorithms working at the pixel level.

With these ideas, by the end of July 2008, I was happy to announce the first draft of G’MIC. The project was officially up and running!

A few months later, in January 2009, enriched by my previous development experience on GREYCstoration (a free tool for nonlinear image denoising and interpolation, from which a plug-in was made for GIMP), and in the hopes of reaching an even larger public, I published a G’MIC GTK plug-in for GIMP. This step proved to be a defining moment for the G’MIC project, giving it a significant boost in popularity as seen below (the project was hosted on Sourceforge at the time).

The sudden interest in the plugin from different users of GIMP (photographers, illustrators and other types of artists) was indeed a real launchpad for the project, with the rapid appearance of various contributions and external suggestions (for the code, management of the forums, web pages, writing of tutorials and realization of videos, etc.). The often idealized community effect of free software finally began to take off! Users and developers began to take a closer look at the operation of the original command-line interface and its associated scripting language (which admittedly did not interest many people until that moment!). From there, many of them took the plunge and began to implement new image processing filters in the G’MIC language, continuously integrated them into the GIMP plugin. Today, these contributions represent almost half of the filters available in the plugin.

Meanwhile, the important and repeated contributions of Sébastien Fourey, colleague of the GREYC IMAGE team (and experienced C++ developer) significantly improved the user experience of G’MIC. Sébastien is indeed at the heart of the main graphical interface development of the project, namely:

• The G’MIC Online web service (which was later re-organised by GREYC’s Development Department).
• Free Software ZArt, a graphical interface - based on the _Qt_ library - for the application of G’MIC filters to video sequences (from files or digital camera streams).
• And above all, at the end of 2016, Sébastien tackled a complete rewrite of the G’MIC plugin for GIMP in a more generic form called G’MIC-Qt. This component, also based on the _Qt_ library (as the name suggests), is a single plugin that works equally well with both GIMP and Krita, two of the leading free applications for photo retouching/editing and digital painting. G’MIC-Qt has now completely supplanted the original GTK plugin thanks to its many features: built-in filter search engine, better preview, superior interactivity, etc. Today it is the most successful interface of the G’MIC project and we hope to be able to offer it in the future for other host applications (contact us if you are interested in this subject!).

The purpose of this article is not to go into too much detail about the history of the project. Suffice it to say that we have not really had time to become bored in the last ten years!

Today, Sébastien and I are the two primary maintainers of the G’MIC project (Sébastien mainly for the interface aspects, myself for the development and improvement of filters and the core development), in addition to our main professional activity (research and teaching/supervision).

Let’s face it, managing a free project like G’MIC takes a considerable amount of time, despite its modest size (~120k lines of code). But the original goal has been achieved: thousands of non-programming users have the opportunity to freely and easily use our image processing algorithms in many different areas: image editing, photo manipulation, illustration and digital painting, video processing, scientific illustration, procedural generation, glitch art…

The milestone of 3.5 million total downloads was exceeded last year, with a current average of about 400 daily downloads from the official website (figures have been steadily declining in recent years as G’MIC is becoming more commonly downloaded and installed via alternative external sources).

It is sometimes difficult to keep a steady pace of development and the motivation that has to go with it, but we persisted, thinking back to the happy users who from time to time share their enthusiasm for the project!

Obviously we can’t name all the individual contributors to G’MIC whom we would like to thank, and with whom we’ve enjoyed exchanging during these ten years, but our heart is with them! Let’s also thank the GREYC laboratory and INS2I institute of CNRS for their strong support for this free project. A big thank you also to all the community of PIXLS.US who did a great job supporting the project (hosting the forum and publishing our articles on G’MIC).

But let’s stop reminiscing and get down to business: new features since our last article about the release of version 2.2!

2. Automatic illumination of flat-colored drawings

G’MIC recently gained a quite impressive new filter named « Illuminate 2D shape », the objective of which is to automatically add lit zones and clean shadows to flat-colored 2D drawings, in order to give a 3D appearance.

First, the user provides an object to illuminate, in the form of an image on a transparent background (typically a drawing of a character or animal). By analyzing the shape and content of the image, G’MIC then tries to deduce a concordant 3D elevation map (“ bumpmap “). The map of elevations obtained is obviously not exact, since a 2D drawing colored in solid areas does not contain explicit information about an associated 3D structure! From the estimated 3D elevations it is easy to deduce a map of normals (“ normalmap “) which is used in turn to generate an illumination layer associated with the drawing (following a Phong Shading model).

This new filter is very flexible and allows the user to have a fairly fine control over the lighting parameters (position and light source rendering type) and estimation of the 3D elevation. In addition the filter gives the artist the opportunity to rework the generated illumination layer, or even directly modify the elevation maps and estimated 3D normals. The figure below illustrates the process as a whole; using the solid colored beetle image (top left), the filter fully automatically estimates an associated 3D normal map (top right). This allows it to generate renditions based on the drawing (bottom row) with two different rendering styles: smooth and quantized.

Despite the difficulty inherent in the problem of converting a 2D image into 3D elevation information, the algorithm used is surprisingly effective in a good many cases. The estimation of the 3D elevation map obtained is sufficiently consistent to automatically generate plausible 2D drawing illuminations, as illustrated by the two examples below - obtained in just a few clicks!

It occurs, of course, that the estimated 3D elevation map does not always match what one might want. Fear not, the filter allows the user to provide “guides” in the form of an additional layer composed of colored lines, giving more precise information to the algorithm about the structure of the drawing to be analyzed. The figure below illustrates the usefulness of these guides for illuminating a drawing of a hand (top left); the automatic illumination (top right) does not account for information in the lines of the hand. Including these few lines in an additional layer of “guides” (in red, bottom left) helps the algorithm to illuminate the drawing more satisfactorily.

If we analyze more precisely the differences obtained between estimated 3D elevation maps with and without guides (illustrated below as symmetrical 3D objects), there is no comparison: we go from a very round boxing glove to a much more detailed 3D hand estimation!

Finally, note that this filter also has an interactive preview mode, allowing the user to move the light source (with the mouse) and have a preview of the drawing illuminated in real time. By modifying the position parameters of the light source, it is thus possible to obtain the type of animations below in a very short time, which gives a fairly accurate idea of the 3D structure estimated by the algorithm from the original drawing.

A video showing the various possible ways to edit the illumination allowed by this filter is visible here. The hope is this new feature of G’MIC allows artists to accelerate the illumation and shading stage of their future drawings!

3. Stereographic projection

In a completely different genre, we have also added a filter implementing stereographic projection, suitably named “Stereographic projection“. This type of cartographic projection makes it possible to project planar defined image data onto a sphere. It should be noted that this is the usual projection used to generate images of “mini-planets” from equirectangular panoramas, like the one illustrated in the figure below.

If we launch the G’MIC plugin with this panorama and select the filter “Stereographic projection“, we get:

The filter allows precise adjustments of the projection center, the rotation angle, and the radius of the sphere, all interactively displayed directly on the preview window (we will come back to this later). In a few clicks, and after applying the filter, we get the desired “mini-planet”:

It is also intruiging to note that simply by reversing the vertical axis of the images, we transform a “mini-planet” into a “max-tunnel”!

Again, we made this short video which shows this filter used in practice. Note that G’MIC already had a similar filter (called “Sphere“), which could be used for the creation of “mini-planets”, but with a type of projection less suitable than the stereographic projection now available.

4. Even more possibilities for color manipulation

Manipulating the colors of images is a recurring occupation among photographers and illustrators, and G’MIC already had several dozen filters for this particular activity - grouped in a dedicated category (the originally named “Colors“ category!). This category is still growing, with two new filters having recently appeared:

• The “CLUT from after-before layers“ filter tries to model the color transformation performed between two images. For example, suppose we have the following pair of images:

Problem: we do not remember at all how we went from the the original image to the modified image, but we would like to apply the same process to another image. Well, no more worries, call G’MIC to the rescue! The filter in question will seek to better model the modification of the colors in the form of a HaldCLUT, which happens to be a classic way to represent any colorimetric transformation.

The HaldCLUT generated by the filter can be saved and re-applied on other images, with the desired property that the application of the HaldCLUT on the original image produces the target model image originally used to learn the transformation. From there, we are able to apply an equivalent color change to any other image:

This filter makes it possible in the end to create HaldCLUT “by example”, and could therefore interest many photographers (in particular those who distribute compilations of HaldCLUT files, freely or otherwise!).

• A second color manipulation filter, named “Mixer [PCA]“ was also recently integrated into G’MIC. It acts as a classic color channel mixer, but rather than working in a predefined color space (like sRGB, HSV, Lab…), it acts on the “natural” color space of the input image, obtained by principal component analysis (PCA) of its RGB colors. Thus each image will be associated with a different color space. For example, if we take the “lion” image below and look at the distribution of its colors in the RGB cube (right image), we see that the main axis of color variation is defined by a straight line from dark orange to light beige (axis symbolized by the red arrow in the figure).

The secondary axis of variation (green arrow) goes from blue to orange, and the tertiary axis (blue arrow) from green to pink. It is these axes of variation (rather than the RGB axes) that will define the color basis used in this channel mix filter.

It would be wrong to suggest that it is always better to consider the color basis obtained by PCA for the mixing of channels, and this new filter is obviously not intended to be the “ultimate” mixer that would replace all others. It simply exists as an alternative to the usual tools for mixing color channels, an alternative whose results proved to be quite interesting in tests of several images used during the development of this filter. It does no harm to try in any case…

5. Filter mishmash

This section is about a few other filters improved or included lately in G’MIC which deserve to be talked about, without dwelling too much on them.

• Filter “Local processing” applies a color normalization or equalization process on the local image neighborhoods (with possible overlapping). This is an additional filter to make details pop up from under or over-exposed photographs, but it may create strong and unpleasant halo artefacts with non-optimal parameters.

  

• If you think that the number of layer blending modes available in GIMP or Krita is not enough, and dream about defining your own blending mode formula, then the recent improvement of the G’MIC filter « Blend [standard] » will please you! This filter now gets a new option « Custom formula » allowing the user to specify their own mathematical formula when blending two layers together. All of your blending wishes become possible!

  

• Also note the complete re-implementation of the nice “Sketch“ filter, which had existed for several years but could be a little slow on large images. The new implementation is much faster, taking advantage of multi-core processing when possible.

  

• A large amount of work has also gone into the re-implementation of the “Mandelbrot - Julia sets“ filter, since the navigation interface has been entirely redesigned, making exploration of the Mandelbrot set much more comfortable (as illustrated by this video). New options for choosing colors have also appeared.

  

• In addition, the “Polygonize [Delaunay]“ filter that generates polygonized renderings of color images has a new rendering mode, using linearly interpolated colors in the Delaunay triangles produced.

  

6. Other important highlights

6.1. Improvements of the plug-in

Of course, the new features in G’MIC are not limited to just image processing filters! For instance, a lot of work has been done on the graphical interface of the plug-in G’MIC-Qt for GIMP and Krita:

• Filters of the plug-in are now allowed to define a new parameter type point(), which displays as a small colored circle over the preview window. The user can drag this circle and move it with the mouse. As a result this can give the preview widget a completely new type of user interaction, which is no small thing! A lot of filters now use this feature, making them more pleasant to use and intuitive (look at this video for some examples). The animation below shows for instance how these new interactive points has been used in the filter « Stereographic projection » described in previous sections.

• In addition, introducing these interactive points has allowed improving the split preview modes, available in many filters to display the « before/ after » views side by side when setting the filter parameters in the plug-in. It is now possible to move this « before/ after » separator, as illustrated by the animation below. Two new splitting modes (« Checkered » and « Inverse checkered » ) have been also included alongside it.

A lot of other improvements have been made to the plug-in: the support of the most recent version of GIMP (2.10), of Qt 5.11, improved handling of the error messages displayed over the preview widget, a cleaner designed interface, and other small changes have been made under the hood, which are not necessarily visible but slightly improve the user experience (e.g. an image cache mechanism for the preview widget). In short, that’s pretty good!

6.2. Improvements in the software core

Some new refinements of the G’MIC computational core have been done recently:

• The “standard library” of the G’MIC script language was given new commands for computing the inverse hyperbolic functions (acoss, asinh and atanh), as well as a command tsp (travelling salesman problem) which estimates an acceptable solution to the well-known Travelling salesman problem, and this, for a point cloud of any size and dimension.

  

  

• The demonstration window, which appears when gmic is run without any arguments from the command line, has been also redesigned from scratch.

• The embedded JIT compiler used for the evaluation of mathematical expressions has not been left out and was given new functions to draw polygons (function polygon()) and ellipses (function ellipse()) in images. These mathematical expressions can in fact define small programs (with local variables, user-defined functions and control flow). One can for instance easily generate synthetic images from the command line, as shown by the two examples below.

Example 1

$ gmic 400,400,1,3 eval "for (k = 0, k<300, ++k, polygon(3,u([vector10(0),[w,h,w,h,w,h,0.5,255,255,255])))"

Result:

Example 2

$ gmic 400,400,1,3 eval "for (k=0, k<20, ++k, ellipse(w/2,h/2,w/2,w/8,k*360/20,0.1,255))"

Result:

• Note also that NaN values are now better managed when doing calculus in the core, meaning G’MIC maintains coherent behavior even when it has been compiled with the optimisation -ffast-math. Thus, G’MIC can be flawlessly compiled now the maximum optimization level -Ofast supported by the compiler g++, whereas we were restricted to the use of -O3 before. The improvement in computation speed is clearly visible for some of the offered filters !

6.3. Distribution channels

A lot of changes have also been made to the distribution channels used by the project:

• First of all, the project web pages (which are now using secured https connections by default) have a new image gallery. This gallery shows both filtered image results from G’MIC and the way to reproduce them (from the command line). Note that these gallery pages are automatically generated by a dedicated G’MIC script, which ensures the displayed command syntax is correct.

  

This gallery is split into several sections, depending on the type of processing done (Artistic, Black & White, Deformations, Filtering, etc.). The last section « Code sample » is my personal favorite, as it exhibits small animations (shown as looping animated GIFs) which have been completely generated from scratch by short scripts, written in the G’MIC language. Quite a surprising use of G’MIC that shows its potential for generative art.

• We have also moved the main git source repository of the project to Framagit, still keeping one synchronized mirror on Github at the same place as before (to benefit from the fact that a lot of developers have already an account on Github which makes it easier for them to fork the project and write bug reports).

7. Conclusions and Perspectives

Voilà! Our tour of news (and the last six months of work) on the G’MIC project comes to an end.

We are happy to be celebrating 10 years with the creation and evolution of this Free Software project, and to be able to share with everyone all of these advanced image processing techniques. We hope to continue doing so for many years to come!

Note that next year, we will also be celebrating the 20th anniversary of CImg, the C++ image processing library (started in November 1999) on which the G’MIC project is based, proof that interest in free software is enduring.

As we wait for the next release of G’MIC, don’t hesitate to test the current version. Freely and creatively play with and manipulate your images to your heart’s content!

Thank you, Translators: (ChameleonScales, Pat David)