Removing Orange Mask from Colour Negative Scans

[carllooper]

Another way of doing the conversion back to RGB (gives exactly the same result) is to make a comp like before, called RGB, with the following comps in it:

CYAN
MAGENTA REPAIRED
YELLOW REPAIRED

And then add another layer of solid white (at the top). This will be the target result of the comp. At first just white.

To the solid white add the following effects:

1. Effect > Channel > Set Channels
2. Effect > Channel > Invert

Then the settings for 'Set Channel' will be the following:

Source Layer 1: CYAN
Set Red to Source 1's: Luminance

Source Layer 2: MAGENTA REPAIRED
Set Green to Source 2': Luminance

Source Layer 3: YELLOW REPAIRED
Set Blue to Source 3's: Luminance

Source Layer 4: None


The rest is then just creative colour balancing and gamma correction, but where you'll find you have access to a far greater range of colour differentiation.

A trick to colour balancing is to always stay focused on a small part of the image, rather than the whole image, and only ask yourself questions of that small part:

1. Is that part of the image (at which you are looking) in the shadow, midtones or highlights? Then select the appropriate mode (shadows, midtones or highlights) to adjust
2. Is that part too red or too cyan - adjust until you can't tell if it's one or the other
3. Is that part too green or too magenta - adjust until you can't tell if it's one of the other
4. Is that part it too blue or too yellow - adjust until you can't tell if it's one or the other

The important point here is to disregard the effect your balancing act has on the remainder of the image. Ignore the remainder of the image completely. Stay focused on the single part of the image that you are balancing. If you get distracted by the larger image you'll quickly end up all over the shop - adrift at sea without knowing where to paddle. The reason for this close focus will become apparent as you proceed. It will be like magic.

Once you are happy with balancing one small part of the image, then just select another small area of the image and repeat the process. Likewise ignore the effect it has on the remainder of the image. I can't say this enough. If you get distracted by the larger image you'll get lost. Anyway, as you reiterate this process you will start to see, as a happy side effect, the image as a whole automagically evolving towards a full colour image. But don't get distracted. Just note the happy effect. Stay focused on the small areas you are balancing. You'll eventually get an idea of the relationship between the small parts on which you are focused at any given moment, and the larger image that is the real target of your efforts.

You'll also start to appreciate what a good idea it might be to one day photograph a Macbeth colour chart.

Leave gamma correction or level correction until you are happy you have a good colour.

C

[rlorenz2]

+1 for After Effects Tutorial

Thanks for the great information just having trouble implementing in AE

[carllooper]

ORANGE MASK REMOVAL IN AFTER EFFECTS

See this excellent article for the theory on masks in negative film (thanks to JeremyC for providing this link):
http://www.brianpritchard.com/why_colou ... orange.htm

Note that the following pipeline is reusable. After implementing it, save it for later reuse.
When needing to reuse, just right click on any previously imported image sequence and select Replace Footage.
And viola - all done automagically.

1. Import your image sequence

2. Create a new composition called SOURCE.
Select:
Composition > New Composition
To rename any composition select:
Composition > Composition Settings
Drag your imported image sequence (from the project window) into this new composition, ie. into the layer window (to the left of the timeline) of the composition. The image sequence will then occupy Layer 1

3. Create a new composition called CYAN
Drag the SOURCE composition (from the project window) into the first layer of this new CYAN composition.
Select the layer called SOURCE (in the CYAN composition).
Add the following effect to the layer called SOURCE with:
Effect > Channel > Shift Channels
Set the parameters:
Take Alpha from Alpha
Take Red From Red
Take Green From Red
Take Blue From Red

4. Create a new composition called MAGENTA
Drag the SOURCE composition (from the project window) into the first layer of this MAGENTA composition.
Select the layer called SOURCE (in the MAGENTA composition).
Add the following effect to the layer called SOURCE with:
Effect > Channel > Shift Channels
Set the parameters:
Take Alpha from Alpha
Take Red From Green
Take Green From Green
Take Blue From Green

5. Create a new composition called YELLOW
Drag the SOURCE composition (from the project window) into the first layer of this YELLOW composition.
Select the layer called SOURCE (in the YELLOW composition).
Add the following effect to the layer called SOURCE with:
Effect > Channel > Shift Channels
Set the parameters:
Take Alpha from Alpha
Take Red From Blue
Take Green From Blue
Take Blue From Blue

6. Create a new composition called MAGENTA MASK
Drag the CYAN composition into this composition.
Select the layer called CYAN
Add the following effect to the layer called CYAN with:
Effect > Color Correction > Levels
Set the Output White parameter to:
51

7. Create a new composition called MAGENTA MINUS MASK
Drag the following comps into this composition:
Layer 1: MAGENTA MASK (on top)
Layer 2: MAGENTA
Select the MAGENTA MASK layer and select:
Layer > Blending Mode > Subtract
Select the MAGENTA layer and add the effect:
Effect > Channel > Invert

8. Create a new composition called MAGENTA REPAIRED
Drag in MAGENTA MINUS MASK
Add:
Effect > Channel > Invert

9. Create a new composition called YELLOW MASK
Drag the MAGENTA REPAIRED composition in
Select the layer called MAGENTA REPAIRED
Add the following effect to the layer called MAGENTA REPAIRED with:
Effect > Color Correction > Levels
Set the Output White parameter to:
51

10. Create a new composition called YELLOW MINUS MASK
Drag the following comps into this composition:
Layer 1: YELLOW MASK (on top)
Layer 2: YELLOW
Select the YELLOW MASK layer and select:
Layer > Blending Mode > Subtract
Select the YELLOW layer and add the effect:
Effect > Channel > Invert

11. Create a new composition called YELLOW REPAIRED
Drag in YELLOW MINUS MASK
Add:
Effect > Channel > Invert

12. Create a new composition called RGB
Add the following:
Layer 1: Select: Layer > New > Solid
Layer 2: CYAN
Layer 3: MAGENTA REPAIRED
Layer 4: YELLOW REPAIRED
To Layer 1 (otherwise an arbitrary solid colour) add the following effects:
Effect > Channel > Set Channels
Effect > Channel > Invert
Select the following parameters on the Set Channels effect
Source Layer 1: CYAN
Set Red To Source 1's: Luminance
Source Layer 2: MAGENTA REPAIZRED
Set Green To Source 2's: Luminance
Source Layer 3: YELLOW REPAIZRED
Set Blue To Source 3's: Luminance

13. Create a new composition called RGB COLOUR BALANCE
Add in the RGB comp.
Add at least the following effects to the layer called RGB

Effect > Color Correction > Color Balance
Effect > Color Correction > Levels

You might also like

Effect > Color Correction > Exposure
Effect > Color Correction > Shadows and Highlights

Colour Balancing Notes (from previous post)

The rest is then just creative colour balancing and gamma correction, but where you'll find you have access to a far greater range of colour differentiation.

For a particular neg stock/scan you'll want to get some balance that acts as a standard for all the shots in your sequence. So check your balance against a number of shots when aiming for this. And then where there are shots that deviate from the standard, such as under or overexposed shots, or shots that were done in a colour environment different to that which the stock assumed (eg. Daylight vs Tungsten) then you will want to do, in addition to your standard colour balance, a second one, tweaking such on a shot by shot basis.

A trick to colour balancing is to always stay focused on a small part of the image, rather than the whole image, and only ask yourself questions of that small part:

1. Is that part of the image (at which you are looking) in the shadow, midtones or highlights? Then select the appropriate mode (shadows, midtones or highlights) to adjust
2. Is that part too red or too cyan - adjust until you can't tell if it's one or the other
3. Is that part too green or too magenta - adjust until you can't tell if it's one of the other
4. Is that part it too blue or too yellow - adjust until you can't tell if it's one or the other

The important point here is to disregard the effect your balancing act has on the remainder of the image. Ignore the remainder of the image completely. Stay focused on the single part of the image that you are balancing. If you get distracted by the larger image you'll quickly end up all over the shop - adrift at sea without knowing where to paddle. The reason for this close focus will become apparent as you proceed. It will be like magic.

Once you are happy enough with balancing one small part of the image, then just select (look at) another small area of the image and repeat the process. Likewise ignore the effect it has on the remainder of the image. I can't say this enough. If you get distracted by the larger image you'll get lost. Anyway, as you reiterate this process you will start to see, as a happy side effect, the image as a whole automagically evolving towards a full colour image. But don't get distracted. Just note the happy effect. Stay focused on the small areas you are balancing. You'll eventually get an idea of the relationship between the small parts on which you are focused at any given moment, and the larger image that is the real target of your efforts.

You'll also start to appreciate what a good idea it might be to one day photograph a Macbeth colour chart.

[milesandjules]

This technique works great thanks Carl…genius. I love how once the after effects project is setup you can replace the previous footage with the new footage without any more project adjustments. Love it.

[Nicholas Kovats]

Carl,

Would you recommend filming a Macbeth color chart prior to a shot or a group of shots under similar lighting? Would that assist my color correctors? Not that my film work is heavily corrected which becomes something else entirely different. I try to originate the best optical quality in the field.

[carllooper]

Nicholas,

So a colour calibration is a set of colour balance numbers you've derived in your NLE, that will reproduce the known values of the colour chart that was shot, when shot on a particular film stock and scanned through a particular setup. So that whenever reusing the same filmstock and same scan setup you can apply the same colour balance numbers.

If you use the correct colour compensation filters in the field there is no need to change the calibration set up in post, ie. no need to reshoot the chart and derive new colour balance numbers. The filters in the field will do the job.

If you don't use such field filters, and without re-calibrating, you could use the digital equivalent of such field filters, ie. on top of the colour calibration you've already set up. For example, in Photoshop and After Effects there is a filter called PhotoFilter containing filters such as an 85.

However without such a digital filter one might like to create one. So shooting a colour chart under varying lighting temperature conditions (and perhaps under various uses/misuses of filters) could provide for that. But note that either way this still requires that one subsequently take notes during a shoot (note to self: forgot the filter) or otherwise remember the conditions of the shoot (that shot was done on an overcast day), or can otherwise guess the conditions (the angle of the light suggests sunset), so that you can apply whatever correction (or not) on top of the calibration.

Shooting a chart allows you to not have to take notes or guess. Taking notes or guessing allows you to not have to reshoot a chart.

The Macbeth chart is interesting. Not only does it provide a colour reference in terms of specific numbers (useful to computer programmers) it also provides specific colours found in nature (useful to artists). This has it's origins in using nature itself as a colour reference. Which is what you can also do - and that which traditional artists have always used. For example, if you photograph some grass and it comes out purple in your scan then you know it's incorrect. The same when painting on a canvas. You can apply your knowledge of grass (that it has a certain green colour) in order to correct the purple image (or in the case of painting: not create it in the first place).

Nature itself (or herself), and your eyes (both creative and technical), become your calibration tools. Science is firstly an art.

C

[carllooper]

This technique works great thanks Carl…genius. I love how once the after effects project is setup you can replace the previous footage with the new footage without any more project adjustments. Love it.

Your welcome.

So it looks like, from the preliminary tests we did, that the auto-correction built into Roger's transfer unit is already properly taking into account the mask in negs, ie. no need to do any uncorrected scan plus a DIY demask. Or to put it another way: doing it one way, or the other way, didn't seem to provide for any evidence of any more improvement in colour scope. Which is consistent with what one would otherwise assume or expect given Roger's long experience in film transfers.

The DIY demasking will be of most obvious benefit when building or using one's own DIY neg scanner.

C

[woods01]

Just wanted to say thank you for this great post and thread. I've been trying to tackle the orange mask problem with a DSLR mounted to a JK optical printer with mixed results. You've inspired me to revisit this task with a fresh mind, thank you!

[carllooper]

Just wanted to say thank you for this great post and thread. I've been trying to tackle the orange mask problem with a DSLR mounted to a JK optical printer with mixed results. You've inspired me to revisit this task with a fresh mind, thank you!

Your welcome. The origin of the thread was in relation to a recent project with which I was involved. A music clip shot on Super16 negative (500T). I was to do the transfer of such to digital on a DIY setup (DSLR + 16mm Analysis Projector). I'd only previously used reversal on such a setup. I was asked if I knew how to remove the orange mask from neg and I assumed I did, but when attempting to do so (following many available tutorials on such) I discovered all of the tutorials must be incorrect. I could not, for the life of me, get the colours I knew must be in the data.

Fortunately I found enough information about the way an orange mask is formed in film to concoct a pretty good digital demask.

The way in which the mask is formed in film (in terms of inverse proportions to the silver>dye image) means that one doesn't have to be particularly accurate in terms of the percentages used in a digital demask. The mask is not a fragile or knife edge solution to the problem of dye impurities. It is 'forgiving' as one might say. So long as one is using a reasonable approximation of the mask (and corresponding demask) the remainder of the task (colour balance) will yield pretty good results. The value chosen for the demask strength (20%) was found to be doing a decent job. But a different value might do a better job. Now one can imagine a more complicated algorithm using filmstock lookup tables (that might be able to squeeze a few more drops of colour out of the demasking stage) but it should be noted that in film-to-film printing no such opportunity would exist: one can only use printer lights and global filters (the equivalent of digital colour balance) to accommodate any masking differences between different neg stock and print stock, yet perfectly good results are still obtainable. In other words it's not necessary to find a demasking control value for a particular film stock (although it wouldn't hurt) - only a value that is an average across a range of film stocks - because at the very least one would obtain a result no worse than that obtainable in a film-to-film print. Now whether that average value is 20% I can't really say, but it worked well enough for Kodak 500T. In any case, just as in film-to-film printing using printer lights, the rest is just done in digital colour balance.

Having said that, if a digital demask can go further than the colour obtainable in film-to-film printing would, that can be something to look into: an algorithm with control values (or even LUT gradients) in yellow and magenta, precisely tuned to a particular filmstock.

C

[Pj]

Maybe I'm doing something wrong, I've never had any major issues correcting properly exposed colour negative film and don't see it as a complicated matter, if anything I have found working with reversal tricky as it can be so contrasty. My method is simple, I have it explained on a PDF on my website;

www.lightbreeze.co.uk

Pav

[carllooper]

From what I understand, the original designers of dye masking might have designed it in such a way that it all it would do was just correct for the particular over-absorption of light in any dye used, and no more, leading to what would then be a flat bias in the negative. And this would also cancel out in any print. And had this design been implemented in film it would then be just a simple flat filter to remove such a bias from any digital scan of such a negative, ie. in the way that many recommend doing it.

But from what I've researched (and it makes sense) it became a far more robust design in film dye masking to over-compensate for the over-absorption in the dyes. It still cancelled out in the same way (during printing) but this over-compensation led to a much stronger bias in the negative, and a correspondingly non-flat bias that otherwise complicates digital scans. However in doing it this way it meant a far better exchange of colours in film to film printing (which prior to digital was all that mattered). Instead of both films holding onto each other by their finger tips, they would hold onto each other by a solid hand grip (or arm grip). Any noise in the reciprocating biases would be more cleanly cancelled. In the same way that the more times you flip a coin the less difference there becomes between the accumulating head and tail count, over the entire count.

So come the digital age, a digital demask on negative becomes required, but one that would need to account for a non-flat bias, rather than just a flat bias (had masking been otherwise designed), the result of such non-flat filtering being a stronger range of colours with which to work in the result. A cleaner signal. Less colour noise. The result becomes far more flexible - open to greater control across the entire tonal range. At least that's been my experience when trying it both ways: demask + colour balance vs colour balance only. It certainly saved my arse on a recent project. Without the demasking I could only get shit results, no matter how many hours I spent in colour balancing. And I spent those hours! The demasking unlocked a whole range of colours that just weren't otherwise extractable with a flat filter.

Necessity was the mother of invention, on the heels of assumption as the mother of all fuckups.

C

ps. If a scanner already implements a demasking algorithm, the RGB numbers for a simple flat filter (or RGB light adjust) will be all that one needs to find. No need to demask. Or rather: doing so would lead to worse results insofar as demasking would end up being applied twice, degrading the signal. Insofar as I had built a DIY scanner without any hidden demasker, I wasn't running that risk. I just had to build a demasker in the first place.

[carllooper]

For the mathematically inclined this is a model of how I understand it. And would form the basis of a more finely tuned plugin.

The mask (m) is inversely proportional, by some compensation factor (k) to the over-absorption amount (a) of the dye in the negative (where the over absorption amount of the dye is proportional to the amount of dye). This can be expressed as:

m = k * (1-a)

The resulting bias (b) in a negative is then given by:

b = a + m

From this we can see that if the mask's compensation factor (k) were 1, then the resulting bias (b) in the negative would indeed be a flat signal (all ones), and therefore removable with a flat filter:

k = 1
a + m = b

0.0 + 1.0 = 1
0.2 + 0.8 = 1
0.4 + 0.6 = 1
0.6 + 0.4 = 1
0.8 + 0.2 = 1
1.0 + 0.0 = 1

But if the mask's compensation factor (k) is not 1, (which is how I understand it) - for example it was k = 2, then the bias would not be a flat signal:

k = 2
a + m = b

0.0 + 2.0 = 2.0
0.2 + 1.6 = 1.8
0.4 + 1.2 = 1.6
0.6 + 0.8 = 1.4
0.8 + 0.4 = 1.2
1.0 + 0.0 = 1.0

It is this non-flat bias in the negative that motivates the development and use of a non-flat filter in any digital scan of the negative.

C

[milesandjules]

Hey carl…why not post the orange mask ae project here so that people can just replace their footage….I know it helped me when you talked me through it.

[carllooper]

Hey carl…why not post the orange mask ae project here so that people can just replace their footage….I know it helped me when you talked me through it.

The version of After Effects I have may not be the same as that which others have. I'm always on the latest version (through a cloud subscription) so it's easier and perhaps more informative that I describe how to make the project from scratch, rather than posting any specific project I've put together. The effects used are ones that have been in versions of After Effects for as long as I can recall, ie. it's as generic as possible.

The following link is to the previous post, in this thread, where I talk through how to put together a reusable After Effects Project that implements the discussed demask:

viewtopic.php?f=1&t=24780&start=30#p211545

Myself, I'm actually more interested in the film to film pipeline (analog printing) - the digital side is more just a way of playing around with and testing theories in relation to the analog side, ie. it's not, per se, the development of some process that can do a good film to digital copy (although that is, of course, a very happy byproduct). The real goal of such (for me) is to take such theoretical play back into the analog domain, ie. where there would be no digital byproduct of such at all. Zero. So it's towards an understanding of the film itself (and film printing), from which this discussion ultimately finds it's destination. Is the theory 'correct' rather than just: 'does it give good digital results'? Is there a better way of theorising film printing (in pictures, words, numbers, software) that will open up more possibilities in how one might then manipulate film printing, ie. in the analog domain. The film printer as analog computer, performing analog computations. The analog automaton.

At the Artist Film Workshop we have a contact printer, restored by Richard Tuohey, and I've also put together an ad hoc but completely functional optical printer (that also doubles as a digital scanner) and that we've used there (but currently back in my home studio for a specific project). At the workshop we're far more interested in art/experimental cinema (on celluloid film) where it's not so much, or rather not at all, about reproducing a given signal in some notionally 'correct' way as manipulating those signals towards more alternative ends. Towards the creation of art, rather than how to deliver some notional service! For example, one might begin with completely arbitrary black and white sources, multi-printing such onto colour stock, through different printer lights and filters, etc. In one direction will be the pleasure (or pain) of the physical results from such, and in another direction, the pleasure (or pain) of theorising those results: be it from an art/theory point of view, or a purely technical point of view, or some connected/disconnected interplay between of the two. Or in some other direction altogether. At a fundamental level anything goes - freedom being the name of the game. No client (or indeed self) expectations to satisfy at all.

C

[carllooper]

Having analysed the situation a little more carefully I've come up with a more tuneable algorithm.

This could be implemented in After Effects as well, but for me it's actually a lot easier just to write it up in code. And if you can read the code you can understand better what it otherwise models and does.

So the following functions are written in a dialect of Javascript, and models the activity of film emulsion in terms of exposure, development and printing/projection.
The process gives completely symmetrical results, ie. if a light signal is run through the functions twice (once to simulate neg stock, and again to simulate print stock) it will come out the other end as the original signal. Between the two is what the digital scan would see. The following is an example output using a white print light = 2 across the RGB channels.

Row 1: the colour of the light exposing the negative
Row 2: the colour emerging from white light projected through neg (what a neg scan sees)
Row 3: the colour emerging from white light projected through a print, made from the neg

0.00, 0.00, 0.00
2.00, 1.80, 1.60
0.00, 0.00, 0.00
----
0.20, 0.20, 0.20
1.80, 1.60, 1.40
0.20, 0.20, 0.20
----
0.40, 0.40, 0.40
1.60, 1.40, 1.20
0.40, 0.40, 0.40
----
0.60, 0.60, 0.60
1.40, 1.20, 1.00
0.60, 0.60, 0.60
----
0.80, 0.80, 0.80
1.20, 1.00, 0.80
0.80, 0.80, 0.80
----
1.00, 1.00, 1.00
1.00, 0.80, 0.60
1.00, 1.00, 1.00
----

For fine tuning a description of a particular filmstock, the following control variables (which have no affect on signal symmetry) can be tuned. They represent the extent to which a dye deviates from an ideal dye, so a value of zero would describe an ideal dye. The values below are just ball park but would work well enough. Magenta dyes are less pure than cyan dyes. So in the following the magenta dye impurity (or 'bleed') has a higher value (ie. more bleed). The cyan dye bleed less so. The yellow dye in real film stock is assumed by mask designers to be near enough to ideal as to require no corresponding cyan mask in real world film.

Code: Select all

MAGENTA_DYE_BLEED = 0.4
CYAN_DYE_BLEED = 0.2	

By default the following mask strengths are made equivalent to the corresponding assumed dye bleed. However this could be adjusted (in conjunction with printer light values) in order to more closely model real world film:

Code: Select all

YELLOW_MASK_STRENGTH = MAGENTA_DYE_BLEED
MAGENTA_MASK_STRENGTH = CYAN_DYE_BLEED

This models the exposure of film to light, be it camera stock in a camera, or print stock in a printer. The density of the silver in each layer is proportional to the mount of light exposing the layer. Basically no more than a name change here. Analog transform: the density of the silver is analogous (or equal) to the light falling on the silver halide crystals.

Code: Select all

 public function exposeTo(light:Light)
		{
			// silver density is proportional to the amount of light
			blueSensitiveSilver 	= light.blue;
			greenSensitiveSilver 	= light.green;
			redSensitiveSilver 	= light.red;
			
		}

This models what happens in development.

Code: Select all

public function develop()
		{
			// dye density is proportional to silver density
			yellowDye = blueSensitiveSilver;
			magentaDye = greenSensitiveSilver;
			cyanDye = redSensitiveSilver;
			
			// mask density is inversely proportional to dye density
			yellowMask 	= YELLOW_MASK_STRENGTH * (1.0 - magentaDye);
			magentaMask    = MAGENTA_MASK_STRENGTH * (1.0 - cyanDye);
			
		}

This models what happens when a printer source light, or projector source light, is projected through the above developed film:

Code: Select all

public function filtering(sourceLight:Light):Light
		{
			// insofar as the dyes are not pure 
			// they affect more than just the source light (printer/projector) that they would otherwise control
			
			var filteredLight:Light = new Light();
			
			// start with source light unaffected by film
			filteredLight.red = sourceLight.red;
 			filteredLight.green = sourceLight.green;
			filteredLight.blue = sourceLight.blue;
			
			// remove from light that which is controlled by cyan dye
			filteredLight.red -= cyanDye;
			filteredLight.green -= (cyanDye * CYAN_DYE_BLEED);
			
			// remove from light that which is controlled by magenta dye
			filteredLight.green -= magentaDye;
			filteredLight.blue -= (magentaDye * MAGENTA_DYE_BLEED);
			
			// remove from light that which is controlled by yellow dye
			filteredLight.blue -= yellowDye;
			
			// remove from light that which is controlled by the two masks
			// the masks themselves would have some bleed, but not modelled here
			filteredLight.blue -= yellowMask;
			filteredLight.green -= magentaMask;
			
			// return the remaining light
			return filteredLight;
			
		}

And the following function is now the 'demasker' which takes filtered light (eg. that from a neg scan) and assumes such light to have undergone the above process (ie. in reality)
and now just runs that same light back through the same process, but in a virtual version of such, ie. reconstructing how printing and developing the print stock would then appear
as light when projected. The original signal exposing the neg stock is reconstructed.

Code: Select all

public function defiltering(sourceLight:Light, filteredLight:Light):Light
		{
			// this is the inverse of filtering and otherwise called "demasking"
			// it is none other than taking the result of filtering 
                       // and running it back through the same expose-develop-project process
			
			this.exposeTo(filteredLight);
			this.develop();
			var defilteredLight:Light = this.filtering(sourceLight);
			
			return defilteredLight;
			
		}