Removing Orange Mask from Colour Negative Scans

[Hansha]

And according to some other source the procedure is not correct ...
http://www.c-f-systems.com/Docs/Negativ ... FS-244.pdf
start reading at pg. 9

Not trying to diminish the work of the authors above, just add to the, by now confusing, debate ...

[carllooper]

And according to some other source the procedure is not correct ...
http://www.c-f-systems.com/Docs/Negativ ... FS-244.pdf
start reading at pg. 9

Not trying to diminish the work of the authors above, just add to the, by now confusing, debate ...

Not to diminish the author of the PDF's work, but I've found what they say is not correct. The mask is not "equivalent to a uniform coloured filter (etc)". The mask varies in inverse proportion to the dye record. So unless you are exposing a uniform scene (such as a grey card or white sheet of paper) the mask is not uniform at all. One can get adequate results assuming a flat mask, but one can get much results by assuming it's not a flat mask and analysing what it is.

That said, perhaps they have just summarised their approach somewhat clumsily. I haven't read through the process they've elaborated. May very well be just fine. The way I've done it is to just follow the physics of it. My own way of summarising could very well be regarded as clumsy as well. And they may very well arrive at the same results as mine but by alternative means.

For me it's just easier to follow the physics.

C

[RCBasher]

Not to diminish the author of the PDF's work, but I've found what they say is not correct. The mask is not "equivalent to a uniform coloured filter (etc)". The mask varies in inverse proportion to the dye record. So unless you are exposing a uniform scene (such as a grey card or white sheet of paper) the mask is not uniform at all. One can get adequate results assuming a flat mask, but one can get much results by assuming it's not a flat mask and analysing what it is.

That said, perhaps they have just summarised their approach somewhat clumsily. I haven't read through the process they've elaborated. May very well be just fine. The way I've done it is to just follow the physics of it. My own way of summarising could very well be regarded as clumsy as well. And they may very well arrive at the same results as mine but by alternative means.

For me it's just easier to follow the physics.

Regardless of all the above comment, I just think back to my early dark room days of making prints from 35mm negatives - a pack of uniform coloured filters was all I had to play with to remove the mask, compensate for the paper batch properties and light bulb colour. This implies to me that the end to end "system" was designed to be used with uniform filters, whether it was optimum or not is another question!

Frank

[carllooper]

Not to diminish the author of the PDF's work, but I've found what they say is not correct. The mask is not "equivalent to a uniform coloured filter (etc)". The mask varies in inverse proportion to the dye record. So unless you are exposing a uniform scene (such as a grey card or white sheet of paper) the mask is not uniform at all. One can get adequate results assuming a flat mask, but one can get much results by assuming it's not a flat mask and analysing what it is.

That said, perhaps they have just summarised their approach somewhat clumsily. I haven't read through the process they've elaborated. May very well be just fine. The way I've done it is to just follow the physics of it. My own way of summarising could very well be regarded as clumsy as well. And they may very well arrive at the same results as mine but by alternative means.

For me it's just easier to follow the physics.

Regardless of all the above comment, I just think back to my early dark room days of making prints from 35mm negatives - a pack of uniform coloured filters was all I had to play with to remove the mask, compensate for the paper batch properties and light bulb colour. This implies to me that the end to end "system" was designed to be used with uniform filters, whether it was optimum or not is another question!

Frank

Yes, I must revisit this and check if I've made some mistake somewhere or otherwise clarify it if I haven't. I know the mask isn't uniform but perhaps there is some shortcut in the maths that can be made by assuming the mask is uniform with respect to an assumption that the dyes are not impure, ie. where two wrongs make a right (as it often does in this sort of thing).

In a neg to print situation one will use a flat filter, or RGB setting, since there is no other way of controlling the colour but I'm assuming this is because the print stock has a corresponding mask to cope with impure dyes in it. I mean it's not as if dyes in a print can be any more pure than dyes in a neg.

If so, then in this situation you would only need a flat filter because the masks in each would otherwise cancel each other out. ie. you only need to adjust for general colour balance rather than mask removal. But maybe that's wrong.

In any case I've never been able to get a good result with a flat filter in a film to digital transfer. It's only when I run it though my non-flat model that I get what seems to me to be good colour data.

However I need to revisit the work because I just can't for the life me remember what I did.

C

[RCBasher]

You could be right in that the print paper could easily have had a non-linear response dyes, the inverse to the film.

[tomfrh]

Carl,

Thanks for this information. It's helping me understand this perplexing question of the orange mask!

A couple of questions:

1. DO photoshop plugins like colorperfect use your method, or do they use simple resetting white/black points method?

2. What do labs typically do when you get scanned negs back? How are they correcting?

3. Is there a fundamental difference between using a scanner vs a DSLR?

Tom

[carllooper]

Carl,

Thanks for this information. It's helping me understand this perplexing question of the orange mask!

A couple of questions:

1. DO photoshop plugins like colorperfect use your method, or do they use simple resetting white/black points method?

2. What do labs typically do when you get scanned negs back? How are they correcting?

3. Is there a fundamental difference between using a scanner vs a DSLR?

Tom

I don't know how colour perfect works or what labs do. My interest in it was to understand the reasons for the mask and how it solves the problem of imperfect dyes - and from a more practical point if view, how to then handle it when digitising negative because in this situation one isn't printing the signal back onto film (where the dye impurities would be there as well, if now inverted with respect to the light).

But to be honest I have to read up on it all again before I can talk about it again. I used this article as a starting point for getting my head around it, and other literature I could find was consistent with this one, but this one remained the one I returned to when working out a model.

http://www.brianpritchard.com/why_colou ... orange.htm

In this article one can see the magenta and yellow coloured masks (together becoming an orange mask) can not be a flat colour. They must form an image. This is assuming that what Brian Pritchard says is correct:

Because the coupler is used to form the dye, the amount of mask will be inversely proportional to the amount of dye formed. That is, if no dye is present then maximum mask is present and in the areas of maximum dye formation there will be no mask.

But is this correct or is it wrong? This is the thing to find out. My understanding is that it's correct, and the model I use for mask removal is based on the assumption that this is correct.

For example, the image on the left shows the cyan channel with it's magenta mask, while the image on the right shows the cyan channel without the magenta mask. If the magenta mask varies in inverse proportion to the density of the cyan dye, then if the magenta mask were separated out, it would form an image - just as the separated out cyan signal forms an image. When the cyan signal and magenta mask are combined together (as they are in film) they would form the image on the left.




--

And below is what the above looks like when taken into Photoshop, converting to CMY, and removing the magenta channel (which was indeed an image - not just a flat signal). There is some residual yellow in the left image because I was removing pure magenta, whereas the dyes in film are impure!




--

And below is what happens if removing the cyan and yellow instead. We see the remaining magenta channel is not a flat signal, but an image. But we can also see that perhaps a flat magenta signal might provide an adequate substitute for this image. One might be able to get mediocre results by assuming the magenta mask is flat.



--

And here I've replaced the magenta channel with a flat signal just to see what it looks like. We can see that a flat signal does provide something somewhat similar to the original. However this is not necessarily indicative of the best way to assume the composite signal was prepared - ie. if one wanted to get the most out of the negative. But it could provide for an otherwise ok result.



That's my theory anyway, and I'm reconvincing myself of it all over again.

C

[carllooper]

I imagine one could design negative film in such a way that it lends itself more readily to digitisation, as distinct from printing back to film, where it would be just varying the amount of red, green, and blue channels of the digital image in a simple way.

But lets think through this - how we might design such. And possibly through this work out why this might not be so.

The purpose of an ideal magenta dye would be to control the amount of green light that will be transmitted through such a dye. The more magenta dye there is the less green light will be transmitted, and the less magenta dye there is the more that green light would be transmitted. Just what we want. And red and blue light would be completely unaffected. Again, just what we want. And the magenta dye would correspond to the amount of green light that originally exposing the film. And indeed, whether pure or not, that is how the magenta layer is formed. It is proportional to the amount of green light exposing the layer in which magenta dye will form.

But insofar as the magenta dye is not perfect it will affect any blue light transmitted through it. It will absorb some blue light when ideally it shouldn't. The more magenta dye there is the more it will affect the blue light. As Pritchard says, this is called "unwanted absorption". The magenta dye is absorbing blue light when it should (ideally) not do so.

So how does adding yellow coupler help this? After all, yellow will have an affect on the blue light - why is this yellow not unwanted absorption?

If we were making film destined for digitisation the answer would be that the amount of yellow we would add would be in inverse proportion to the amount of yellow in the magenta dye, so that the effect on blue light would be everywhere the same, ie. the affect on the blue signal would be flat. No image. The unwanted yellow in the magenta dye would balance with the yellow coupler added (where the magenta is otherwise low). An even amount of yellow would be in the magenta layer.

BUT - and this is where I could be completely wrong: the dyes in print stock must suffer the same impurity that dyes in negative stock suffer. In other words it seems to me (without revisiting this) that one would want to affect the blue light in some way, but only to the extent that it corrects for the impurity of dyes used in the print stock. However, as I think about this - in the case of blue light, it controls the yellow layer in the print stock, but the yellow dye is the least offensive of the dyes, and so the blue signal (mediated by magenta and yellow coupler) should be flat. Would it want to be anything other than flat even if yellow dye was otherwise?

Hmmm. This is where my brain hurts and I otherwise turn to mathematics. What is at issue is not so much neutralisation of unwanted absorption (which makes sense for neg designed for digitisation) but what we would want to do (other than neutralisation) to make up for impurities in print stock dye. Hmmm.

Anyway lets turn to the other dye/coupler pair.

Cyan dye + magenta coupler.

The cyan dye ideally controls only the red light (and is sensitised by red light). But the cyan dye, insofar as it is impure, also affects green light. So it has an unwanted affect on green light, caused by a magenta bias in the cyan dye. What would this unwanted variation in green have on the signal? Green light controls the amount of magenta dye in the print. So this magenta bias in the cyan would interfere with what the magenta dye in the print should be. By adding magenta coupler in inverse proportion to the cyan dye in the negative, we can completely neutralise the affect it has on green light. And therefore it's affect on the magenta dye.

... to be continued when my brain is working ...

C

[tomfrh]

Thanks for your answers. You've helped me understand a lot about how this all works. I've been through your photoshop method now, and can do it quite quickly. My understanding is still very limited and fuzzy however. Those film chemists were smart guys! It's like magic.

I think that fundamentally you are right, but that in practice level fiddling gets people close enough for their purposes. I'd prefer to find the "true"colours though, if only as a challenge!

I have a couple more questions:

1. The removal doesn't quite eliminate the orange cast, as the values are approximate. How would you remove it perfectly? I.e. how to translate the amount of yellow/magenta into corretion factors?

2. The change to CMY appears to degrade/shift the colours a little, even doing nothing else. Is there any way to avoid this? Could you keep it in RGB only? My poor brain trips at the first hurdle trying to conceive of it!

All this colour stuff does my head in. The other day when I realised magenta is "non-spectral" and doesn't exist in nature I practically fainted with confusion!

[carllooper]

The other day when I realised magenta is "non-spectral" and doesn't exist in nature I practically fainted with confusion!

Funny you should say that. I was told that recently and I thought "hold on, is that right" and in a sense it is. Magenta doesn't exist as a single wavelength of light, whereas all the other colours can.

However that doesn't necessarily mean magenta doesn't exist in nature. We can superimpose fundamental waves (such as red and blue) to make a more complex wave (such as magenta), but why does that mean the more complex wave doesn't exist in nature? We have to ask ourselves why fundamental waves should be considered as any more existant (in nature) than a composite wave. It becomes a question which magenta can provoke.

As for the other questions I'll have to revisit what I've elaborated before I can answer. I tried to, in my last two posts, but my brain is just failing me at the moment. I'm half convinced I've made some mistake somewhere and I'm trying to find it or otherwise reconvince myself its' not there.

C

[tomfrh]

Well, the other "spectral" colours really do have corresponding physical reality, and even though we know the impression of "red" is in fact in our brains, we can at least take solace in the fact that there is a unique frequency out there in the REAL WORLD that it corresponds to.

but with these non spectral colour, e.g. magenta, our brain is totally misleading us, telling us there is a colour, but in fact there is no corresponding colour in the real world!

White/grey/black too! There's no such thing as "white".

[tomfrh]

Ok, I read through your recent posts, and Pritchards commentary, and am starting to get it (I think!), and I want to write it down before I forget it:

The Magenta layer should only catch Green, but also catches some blue. The more magenta, the more unwanted blue caught.

So, if we add yellow (which catches only blue), in inverse proportion to the magenta layer (our yellow mask) - we catch a UNIFORM amount of unwanted blue across the image.

Similarly, the Cyan layer should catch only red, but also catches green. The more cyan, the more green caught.

So if we add a magenta mask to the cyan layer....we catch a uniform amount of green.

So whilst the masks themselves are not simple colour biases, the masks PLUS the corresponding dye impurites are a simple colour bias - yes?

Dye impurity + coupler = uniform colour shift

Which makes me wonder - why then reconstruct the masks? why not simply correct for the uniform amount of unwanted colours? I.e. correct for the dye impurity and mask in one hit?

[carllooper]

Ok, so I don't think I've made any error in the method I've elaborated (but don't quote me) but I'm thinking there might be an error in the implication that one can't otherwise accomodate for the mask by a simpler method.

The method I use is based on the idea that one should disentangle the magenta and cyan dye signal, from the magenta and cyan coupler signal, by using the yellow signal as a key (since the yellow dye signal has the least affect on the light it shouldn't otherwise affect, ie. it doesn't pollute the red and green printer light). It controls (as it should) only the blue light.

However what is now dawning on me is that a coupler must surely completely balance the bias in the dye to which it is coupled. So for example, if cyan dye has a magenta bias (as it does), then the magenta coupler in the cyan dye should be (I assume) such that it does completely neutralise the bias. For if it didn't do so there would be a faint copy of the original red signal (otherwise correctly encoded in the cyan dye layer) that would be echoed in the printer's green light (after passing through the dye). And I can't see how anything other than complete neutralisation of such a green signal (as an echo of the red signal) would help anything downstream from this (such as dye impurities in the print stock).

In other words, while the orange mask is an image (not flat) it must be an image that is the exact opposite of the bias in the magenta and cyan dyes (equally an image - not flat, but the opposite image). In other words, what will be flat, (if this theory is correct) is not the mask, or the bias, but the two combined: the neutralised signal (dye bias + coupler counter bias). They would cancel each other out, at the expense of a change in green/blue levels, but everywhere equal, ie. flat. If so, (and it seems to me it must be so) it becomes just a question of bracketing out this flat signal: in the green channel (from magenta bias minus magenta coupler), and in the blue channel (from yellow bias minus yellow coupler). The red light signal should be just fine.

And then after doing that one just does an invert.

Will need to test this theory out. If so it would be a lot simpler than the previous method. But it leaves unanswered (for me at this current moment) how dye impurites in print stock are resolved. It feels like a shortcut that isn't yet fully resolved as to what else might be needed when otherwise using an RGB filtered sensor rather than print stock. Or to put it another way - if it otherwise works for a sensor, how then does a print in film not end up reintroducing pollution with it's dyes?

I've assumed (and this is probably correct) that the dyes in print stock must also have a coupler. The model I developed is certainly based on this assumption. But I also assumed the signal between neg and print must be skewed in a non flat way in terms of its colour, simply because the masking is not flat. But then the dye bias is not flat. And it's difficult to see how one could get any benefit from one not neutralising the other. However the model was developed independantly of this apparent need for neutralisation (or lack thereof). The model, when unpacked, could very well demonstrate this neutralisation itself, leading to a more compact model. Or the model may be flawed (whichever way the cookie crumbles).

It may simply be I'm getting good results purely during the colour balance stage - by my colour grading skills - by eye - rather than anything in the algorithm.

C

[carllooper]

Ok, I bit the bullet and went back over the code I wrote (rather than the After Effects model of which I'm not 100% certain) and the code checks out.

Indeed I can see in the code that the mask (the coupler signal) does indeed cancel out the dye bias.

It's actually defined by these two lines, where they are given the same value, and where they are otherwise employed in the code they cancel each other out. The 'bleed' value here is what I've since been calling dye bias.

YELLOW_MASK_STRENGTH = MAGENTA_DYE_BLEED;
MAGENTA_MASK_STRENGTH = CYAN_DYE_BLEED;

And the signal reconstruction is perfectly reconstructed when assuming the print stock is the same as the neg stock, ie. both with the same dye bias and same strength of dye couplers. When otherwise emulating print stock for a neg scanner signal the same obtains. A perfect reconstruction. The only vagaries are choice of values for dye bias. The code automatically makes the mask inversely proportional to the chosen dye bias.

The model just follows the way I understand the mask is implemented, and the way light and filters otherwise work, and it produces the correct result when I run it, ie. a correct reconstruction. If I were to remove the couplers in the print stock from the model it fails. So this seems to indicate that it's couplers in the print stock that resolve the dye bias there. The signal between neg and print is an otherwise clean signal with a flat bias in the green and blue channels.

The model reduces the problem to deciding on a value for the magenta and cyan dye bias: just two numbers. So in principle one could have two sliders controlling these variables on the model, through which the signal is otherwise processed, and tweaking the image using such sliders, and judging the result by eye. And that this is otherwise equivalent to adjusting the green and blue channel on a neg scan, which I hadn't, until now, quite appreciated.

So it looks like I'm in the clear in terms of the model I was using, but so too are the authors of the PDF !

Without reviewing the After Effects model, I recall the cancellation was done there as well - so long as one chooses good values for the dye bias. But otherwise, it seems to me, adjusting the green and blue channels of a scan should also work (contrary to what I previously assumed). And it would be a whole lot simpler. But that said, I've never been able to get a good baseline just twiddling those. But now that I'm understanding the whole thing a little better perhaps I could do so.

C

[carllooper]

Ok, I read through your recent posts, and Pritchards commentary, and am starting to get it (I think!), and I want to write it down before I forget it:

The Magenta layer should only catch Green, but also catches some blue. The more magenta, the more unwanted blue caught.

So, if we add yellow (which catches only blue), in inverse proportion to the magenta layer (our yellow mask) - we catch a UNIFORM amount of unwanted blue across the image.

Similarly, the Cyan layer should catch only red, but also catches green. The more cyan, the more green caught.

So if we add a magenta mask to the cyan layer....we catch a uniform amount of green.

So whilst the masks themselves are not simple colour biases, the masks PLUS the corresponding dye impurites are a simple colour bias - yes?

Dye impurity + coupler = uniform colour shift

Which makes me wonder - why then reconstruct the masks? why not simply correct for the uniform amount of unwanted colours? I.e. correct for the dye impurity and mask in one hit?

Thanks tom,

Yes. I totally agree. Your post on this was delayed by the moderator so I didn't get to see it until after I had posted the same conclusion. But yes, we've reached the exact same conclusion.

The computer model I wrote was written to see if it was consistent with the After Effects model, but I didn't actually go through and check the After Effects model with it. I just ended up using the computer model. I think the After Effects model must be wrong, or it's working despite the assumptions driving it, or it doesn't do anything useful at all and the only reason I've ever got any useful result with such was purely during the colour balance after that - possibly undoing errors produced during the After Effects process. But I must go back and review that.

However the computer model works, and as I mentioned in subsequent posts, it' has been modelling exactly what you've otherwise suggested in your post: that dye bias + coupler = flat bias. I just hadn't realised it!

So my conclusion is that the best method is just to adjust the levels on the green and blue channels by eye (with the help of histograms on each) and then do a colour balance by eye - to neutralise any overt residual colour bias. This latter step being the more crucial one in terms of subtle results.

The technique I use in colour balancing is to try and remove all color from the image!

For example if it's too blue I shift the blue/yellow slider towards yellow. Then if it's too yellow I shift it back towards blue by half that amount. Eventually when I can't see either too much blue or too much yellow, I leave that slider where it is and work the other ones: red/cyan and green/magenta.

By trying to remove all colour from the image I end up with the opposite result

C