Tech Feature: HDR Lighting

Introduction

Hello my name is Peter and I’m the new graphics and engine programmer. New is not really the correct word since I have been working at Frictional for a year now. During this time I have updated the engine and added a lot of new graphic features. This will be the first of my blog posts descripting the changes that have been made.

HDR

One of the biggest changes to the the new engine is the introduction of HDR (High Dynamic Range) Lighting. This is a technique to increase the detail of the lighting system. The benefit of using HDR is that bright things can be really bright, dark things can be really dark, and details can be seen in both.

In nature there is no limit to how bright something can be. The difference between a 60 W light bulb and sunshine hitting the earth is around 10 000 luminance (cd/m^2). This means that we need a way to store high intensity values while keeping the quality and precision of the dark areas. Thankfully there already exists a method for storing such values - by using floating point numbers.

We use a 16-bit fp RGBA buffer to store our lighting. This gives us enough of a dynamic range without taking up too much memory.

Tone mapping

A normal computer monitor can display 8-bit colors between the value of [0..1]. Because of this the monitor can not display a 16-bit HDR image directly. To be displayed the image will have to be converted to 8-bit while keeping as much of the details as possible.

Tone mapping is the process of converting an image of dynamic range to one with a clamped range between [0..1].

The simplest method for doing this is to use the Reinhard tone mapping algorithm.

vec3 color = x / (x + 1.0)

No matter how high x gets the final value will always stay between [0..1]. The problem with Reinhard is that it desaturates your dark colors and removes contrast. In the brighter parts of the image Reinhard produces great result with its soft highlights.

What you want to have is an algorithm that preserves the saturation of the color in the dark areas and that keeps as much of the contrast as possible.

I ended up choosing an algorithm created by John Hable for Uncharted 2.

vec3 Uncharted2Tonemap(vec3 x)
{
float A = 0.15;

float B = 0.50;

float C = 0.10;

float D = 0.20;

float E = 0.02;

float F = 0.30;

return ((x*(A*x+C*B)+D*E)/(x*(A*x+B)+D*F))-E/F;

}

Left: Long curve, Right: S-curve near dark values

This algorithm is based on a filmic tone mapping algorithm created by Kodak. It keeps the nice highlights from Reinhard while using a slightly S-shaped curve too keep the dark colors saturated.

Exposure

If you have been sitting in a dark room for some time and walk out into the sun, your eyes will not be ready for the bright light and you will have to squint. After a while your eyes will have adjusted to the light and it will not bother you anymore.

Exposure is a way to control the intensity of light that gets passed through the lens. In the eye this is controlled by the size of the pupil and in a camera it is done by selecting for how long the sensor should be active.

There are a few different ways to control exposure in a game. It can be controlled automatically by storing the average luminance over a few frames and calculating a exposure from that value.
We choose to go with a much simpler method that lets the artists control the exposure by dividing the level into areas that have different exposures. So in a dark area the exposure can be increased and in an outdoor area the exposure can be decreased.

vec3 color = Uncharted2Tonemap(scene_color * exposure)

White Point

A white point is used to increase the contrast of the image. This is the value that is selected to be the brightest any pixel can be. Pixels brighter than the white point will be clamped to 1.0.

vec3 color = Uncharted2Tonemap(scene_color * exposure) / Uncharted2Tonemap(white_point)

HDR Bloom

When a pixel is brighter than the white point it can be used to generate an additional post effect called HDR Bloom. Pixels that get too bright should start bleeding over to other nearby pixels, producing a halo around them. It is a subtle effect that adds realism to the image.

To solve this I use a few render passes. First a bright pass is applied to the image which removes all pixels that are below the white point and scales down all the pixels that are above it. The result is then blurred to create the halos. The blurred image is then added to the original image. The HDR Bloom effect must be performed before the tone mapping.

Final Thoughts

I would say that HDRL and filmic tone mapping is the most important part of any rendering pipeline. It greatly increases the quality of lighting and will make your game look much more realistic.

But HDR and tone mapping is all for nothing if your calculations are not done in linear color space. My next tech feature will focus on gamma correction and the linear color space.

References

http://filmicgames.com/archives/190

http://filmicgames.com/archives/183

http://filmicgames.com/archives/75

http://http.download.nvidia.com/developer/presentations/2004/6800_Leagues/6800_Leagues_HDR.pdf

Who can sit next to children on flights?

[This post originally appeared on Forbes.com on 14th August 2012].

It seems almost silly to ask the question: who can sit next to children on flights? Obviously, parents would be desirable but sometimes that can’t happen. It may be that a child or children are flying

Virgin Australia's Airbus A330-200 (R) taxis past a Virgin Blue Boeing 737-800 (R) after landing at Sydney International Airport on May 4, 2011. Domestic carrier Virgin Blue and its international offshoots, Pacific Blue and V Australia, will all be known as Virgin Australia, with negotiations underway to bring Polynesia Blue under the same umbrella. (Image credit: AFP/Getty Images via @daylife)

unaccompanied or maybe, because of poor airline seating policies, they are apart from parents (Air Canada, I’m talking about you here). In those cases, they are going to end up seated next to an adult they don’t know.

Can You Outsource Child Care On A Plane?

Joshua GansContributor

O' Air Canada, do you really want to separate families?

Joshua GansContributor

Well, it turns out that some airlines actually have a policy on this. Specifically, it appears that if the adult stranger (to the children) is a women, that’s fine but for a man, that’s prohibited. Story No.1 comes from Virgin Australia. They apparently have this policy and unseated fire fighter, John McGirr, who had been seated next to two children. McGirr recounts the experience:

‘Sir we are going have to ask you to move’

‘Why’, I said.

‘Well, because you are male, you can’t be seated next to two unaccompanied minors’.

Shocked, I replied, ‘ Isn’t this sexist and discriminatory?’

She replied, ‘I am sorry, but that is our policy’.

Basically, he felt like a pedophile precisely because he was treated like one. The story created an uproar (although not everyone disagreed with Virgin’s policy). Virgin are now reviewing their approach.

Virgin are not alone. The week brought a similar incident on Qantas who claimed they were maximizing “the child’s safety and well-being.” Really Qantas? You seem to run out of pre-ordered children’s meals quite quickly if that is really what you are doing.

And here is Boris Johnson’s experience on British Airways. Yes, that is Boris Johnson who is now mayor of London.

There we were, waiting for take-off, and I had just been having a quick zizz. It was a long flight ahead, all the way to India, and I had two children on my left. Already they were toughing each other up and sticking their fingers up each other’s nose, and now — salvation!

Hovering above me was a silk-clad British Airways stewardess with an angelic smile, and she seemed to want me to move. “Please come with me, sir” said the oriental vision.

At once, I got her drift. She desired to upgrade me. In my mind’s eye, I saw the first-class cabin, the spiral staircase to the head massage, the Champagne, the hot towels. 

“You betcha!” I said, and began to unbuckle. At which point, the children set up a yammering. Oi, they said to me, where do you think you are going? I was explaining that the captain had probably spotted me come on board, don’t you know. Doubtless he had decided that it was outrageous for me to fly steerage, sound chap that he was. I’d make sure to come back now and then, hmmm? 

At which the stewardess gave a gentle cough. Actually, she said, she was proposing to move me to row 52, and that was because — she lowered her voice — “We have very strict rules”.

Can You Outsource Child Care On A Plane?

Joshua GansContributor

O' Air Canada, do you really want to separate families?

Joshua GansContributor

Eh? I said, by now baffled. “A man cannot sit with children,” she said; and then I finally twigged. “But he’s our FATHER”, chimed the children. “Oh,” said the stewardess, and then eyed me narrowly. “These are your children?” “Yes,” I said, a bit testily. “Very sorry,” she said, and wafted down the aisle — and in that single lunatic exchange you will see just about everything you need to know about our dementedly phobic and risk-averse society.

Johnson was saved public humiliation but say the injustice in the policy. Subsequently, British Airways was forced to change its policy following a law suit. They now try and sit children together. They apparently think, against all evidence, that an adult is more likely to be a sex offender than a child might be a bully.

Which brings me to my story. I wasn’t discriminated against but having been incensed by all this discussion and the fear of pedophiles, I took a different approach. The other day we boarded our long, 13 and a half hour flight from Sydney to LA. Our family had two seats at an end and then three of the four seats in the middle of our Qantas aircraft. Our plan had been to seat the two eldest children together and have the two adults with the youngest one. The youngest wasn’t too happy about this as she wanted to be next to her brother and sister. But then we got on the plane and sitting in the four seat was a man. That caused me to think about our youngest child’s plea. Why were we doing things that way? It could be we were trying to spare the stranger any discomfort from sitting next to a child. But it could also have been that we subtly held ‘stranger danger’ feelings ourselves? I couldn’t risk the latter so I let the children sit with each other while we parents slinked over to a separate area (in the same cabin but over the continuous mathematical line that would be perfect accompaniment). We would defiantly protest Qantas’ policy.

Now the news is, of course, that nothing bad happened. Well, to our children. To the man, I’m not so sure. It was a pretty smooth flight but you can’t get perfect behavior for 13 odd hours. But he didn’t call a waitress and claim that he should be moved according to Qantas policy so it couldn’t have been too bad. Perhaps he was pleased that some parents had confidently not considered him to be a sex offender.

But think of the alternative. Yes, Qantas might be discriminating against men by making them appear to be sex offenders but, when they get their seating policy right, a disproportionate number of women are finding themselves seated next to unaccompanied children on flights. Boris Johnson’s heart lept when he thought he might be moved from his own children as we know that adults, if given a choice, would rather be next to other adults. And there are many more women suffering from this than men likely being moved. Sounds like another consequence of discrimination to me.

When it comes down to it, like so many aspects of airline policy — consider how you are forced to turn off a Kindle on takeoff — the airlines have got their priorities wrong because they can’t get their statistics right. Consider the caption in this recent British Airways ad.

We test our cheese as meticulously as we test our engines.

If not for the accompanied small font explanation that might be taken in a bad light. We expect airlines to quantify risks well when it comes to safety but surely our confidence in them might be harmed a little when they can’t calculate risks in relation to other matters.

For unaccompanied children, I suspect the risk of any criminal behavior befalling them is might less than say, them being misplaced by an airline. On that score, this recent United Airlines experience may given parents far more cause for concern. 

The Self, Presence and Storytelling

At GDC last week I gave a talk called "The Self, Presence and StoryTelling". There will be a version of it up on the GDC Vault in 2 - 3 weeks I think and hopefully it will be free like last years' talk. Before that comes up I will put up a pdf version of the talk containing a bit more information (something I promised at the end of my talk). You can get hold of that here:

Download "The Self, Presence and Story-telling"

Get the mobi version here (thanks to Tomasz Rozanski for creating this).

The paper is basically a summary about a lot of the stuff that has been written on this blog. It is an attempt to define a new design approach that makes it easier to make games that can deal with a wider range of themes and let you play all the way through. I would be very interesting any feedback you got as I would like to keep the document updated and revised if there are any new insights or any info in it that is no longer true.

EDIT: Proof-read the paper a bit and made a little bit more readable.
EDIT2: Mobi version now available.

Can economics help you find a spouse?

[This post was originally published at Forbes.com on 9th August 2012]

That is the question reported Jessica Irvine put to me a couple of weeks ago. It is the type of question that instills fear in an economist. Why? Because the truthful answer can cause trouble at home.

What is the truthful answer? Here is what economist and now Australian parliamentarian Andrew Leigh told Irvine:

‘‘Economically, the answer to this is easy,’’ says Andrew Leigh, a federal Labor MP awarded the Economic Society of Australia’s young economist of the year award last year. ‘‘You’re solving an ‘optimal stopping’ problem. You know you’ve found ‘the one’ when you determine that the expected quality of all future matches is lower than the value of your current partner.”

You see the problem. Economics tells you that you should eventually settle in your search for a spouse. But for the Economist saying it, doesn’t that imply that he settled as well?

Leigh realized his problem and so immediately went what we could term the ‘sap route.’

In my case, I knew I wanted to pop the question to Gweneth [Leigh’s wife] because she made my heart beat faster each time she entered the room.

At this Olympian time, I believe we can score this using a diving analogy. The whole exercise had a degree of difficulty but there was some splash associated with Leigh’s re-entry. Basically, his answer was completely unrelated to the conundrum. Leigh appears to say that he settled but knows why he settled. By referencing the heart he alludes to abandoning his economic mind but we can’t really be sure.

Now let’s turn to the attempt by Rory Robinson. He is an economist but in the private sector:

A similarly sweet, but matter-of-fact, assessment is offered by Rory Robertson, a leading financial markets economist, of his wife. ‘‘In the process of choosing, my assessment was that my wife had the highest weighted-average of all the things I felt were important: looks, ‘compatibility’, kindness, inherent optimism, competence, diligence, and general enthusiasm for a happy life and family.’’

When I ask if he considers it fortunate his wife also assessed him as having the highest weighted-average of her preferences, Robertson replies: ‘‘Something like that, I’d like to think!’’

Robinson offers a longer CV for his partner’s attributes but ultimately is hedging. Again a splash.

Now you might be interested in what a disaster looks like. If so, let’s watch Tim Harcourt, author of The Airport Economist:

‘The economics of love or marriage is like search theory in the labour market. You start out with preferences about what you may like in a partner, but over time you may, if searching unsuccessfully, drop your ‘‘reservation wage’’ – ie lower your standards – to settle on a job or love match. I have a lot of great, smart, good-looking female friends in Sydney who seemed to have had to reduce their reservation wage over time.”

Oh dear. While he gets points for accuracy, in terms of any ‘at home constraint’ I think this one belly flopped. To be sure, Harcourt may have been quoted out of context but you can see just how difficult this exercise is.

Of course, you are by now wondering how I did at this dive. I guess the fact that I am drawing your attention and my spouse’s attention to it gives the answer. While it wasn’t published my opening set-up was this.

… Dateonomics is about finding the right ‘other parent’ for your future children. Economics has a lot to say about that in much the same way it has a lot to say about finding the right employees for your business.

But you can now watch the actual dive:

Economist and author of ‘‘Parentonomics’’, Joshua Gans, says that in looking for that perfect parent for your unborn child, it doesn’t make sense to wait around for Mr or Ms Perfect.

‘‘In the search for the right [partner], economics tells us that you would almost always settle. You search and there are search costs. A rational person should have an optimal stopping rule and if that rule is to find the perfect match out of 7 billion living people, mathematics tells us you will never stop. Except, of course, in my case where settling turned out to be indistinguishable from optimising! For everyone else, they have settled and so if your spouse isn’t perfect you can at least take comfort in the fact that you saved the costs of shopping around.’’

That last bit was the ‘money quote’ for the article. But it was the middle where I took myself out from the entire population as an exception. My spouse didn’t buy any of it (she is too much of a rational engineer to believe in exceptions) but did offer up an “I love you too dear” for effort. I’ll take that as, “I made it!”

Irvine’s entire piece on Dateonomics will appeal to readers of this blog as will likely her new book, Zombies, Bananas and Why There are no Economists in Heaven. There are no economists in heaven? Makes sense. Most economists I know pretty much adhere to the thought experiments in John Lennon’s Imagine which pretty much rules heaven out.

Camps, Sickness and Smell

[This post was originally published at Forbes.com on 22nd July 2012]

As regular readers are possibly aware, all three of our children (ages 13, 11 and 7) are away at Camp this month. We have been in irregular communication with them consisting of emails sent by us through

Contagion (film) (Photo credit: Wikipedia)

the Bunk 1 service and intermittent letters sent by them through the ordinary post. In our case, we have recounted in detail all of the activities we have been able to undertake (mostly travel) because we aren’t with them. In their case, there is a recount of activities they have been involved in, a description of other kids they like or dislike and then complaints as to the things other children where allowed to bring that we prohibited based on our ‘letter of the law’ reading of camp rules.

This weekend, however, was Visitor’s Day where we can travel a few hours north to visit our children at camp. That was the plan. However, last week, we were informed that our youngest children had succumbed to what appears to have been a major outbreak of stomach flu. They recovered in time for Visitor’s Day but enough other children were still ill that the whole day changed. It was supposed to be a situation where we went to see the camp and what our kids had been up to. Instead, it turned into a highly structured set of manoeuvres whereby we could pick up our children to take them out for a few hours. We had to drive up to a location far from the camp. We could not leave our vehicle. Our children could then walk to the vehicle and get in but not before another dose of hand sanitizer.

As it turned out, our children were well acquainted with the ‘Plague protocols’ that turned out to be closely modelled on the movie Contagion. For a week they had been living in a strictly enforced totalitarian regime. No one could get food or touch anyone without permission. Whole cabins were quarantined. That may have sounded bad but, as it turned out, being sick had the silver lining of a break from camp activities and lounging around watching TV. All in all, this proved an education for the children in germ warfare. That said, it is unclear that it did much good. What it did do is keep the whole plague under wraps. When we took our children into town they were all blissfully unaware of the pox that had hit their area.

The sickness had another advantage. Normally, picking up kids from camp is an unpleasant activity. When we drove our eldest back from New Hampshire last year to Boston, we almost died of the smell. We were prepared for the worst this time around but it turned out that part of the protocols was rigidly enforced washing. That saved us much distress.

Our expectation had been that the kids would not want to see us. As it turned out, the tough week had softened them up and they were all happy to be free of austerity for a little while. While we didn’t get to see camp life we got to hear all about it. I suspect all three will be returning next year.

We were also softened up for all this. Thanks to Bunk1, whose incentives are to ensure that parents never forget their children are at camp so that they will write more emails, we had been peppered with Newsletters about things that can go wrong. Here is one. I enjoyed the tick advisory. That seemed to involved wrapping your children in clothes and giving them a level of organization unprecedented for anyone other than Mr Monk.

Of course, the Bunk1 theme has spread through the Internet. In particular, the nightly search through hundreds of photos for one with your own child. Here is a great video capturing the situation there.

Is it a good idea to communicate with your child at camp?

[This post was originally published at Forbes.com on the 11th July 2012]

For the first time ever, all three of our children (13, 11 and 7 years) are away at Summer Camp. Fortunately, they are all at the same camp which saves us considerable bureaucracy and also provided some comfort that they might be there for each other.

Images from our first Summer Art Camp of the year, "Wild and Wacky Portraits" (Photo credit: Wikipedia)

Of course, near as I can tell our children would prefer to have nothing with each other while at camp. To say that this experience is liberating for us, as parents, is an understatement. We seem to have been transported back to a calmer time and have constantly had to challenge our usual behaviours of intense planning and tight scheduling. Things are very, very relaxed.

Suffice it to say, this was enough to spur our kids to grudgingly make contact with us. The 7 year old was the most enthusiastic and seems to send a letter a day. Her first one said: “Dear Mum and Dad, I miss you very much so I want to stay 4 weeks.” She was signed up for just 2 weeks. While the logic of her argument was flawed, as it turned out our lives were very different without her, so we promptly agreed to the extension.I believe that the same sense of liberation is going on for our kids. We are fairly strict parents and certainly on one key dimension — food — very strict. Camp provides a menu that is supposedly healthy but, in fact, is much more kid-friendly than our children’s usual fare. For instance, they appear to have dessert; a concept our kids dream about. At camp they get it every day. The also have something called ‘tuck’ that allows them to get special treats twice a week. But ‘tuck’ comes with a condition — no letter home, no tuck.

Our eldest wrote a varying set of sentences that were not so much designed to convey information but to be written neatly and cover exactly two pages with neat handwriting. We learned from that letter that the tuck letter writing requirement was a two page letter precisely because she ended the second page mid-sentence and did not even bother to sign the letter. We will likely receive the rest as the two page requirement for another instalment.

Our 11 year old son was the only one to convey actual information including his slow evolution of thought about staying on for two more weeks. He has decided to do so.

Alas, the tuck incentive has led to an imperfect flow of information from our children to us. But, in many respects, that is what the Camp wants. Every detail can be seized upon by parents and it is a thankless task.

But what about the flow of information in the other direction. Our Camp uses a service called Bunk1. That service gives parents the opportunity to pay in order to write emails that are printed out and delivered to children. It isn’t cheap and it requires being at a computer to use (I guess you could do it on a phone but it is hard). According to this Time article about ‘kid sick’ parents, Bunk1 was founded by Ari Ackerman to provide a “one way window” into the Camp world. With these emails I guess we can throw things through that window.

What do you write to a child on camp? Our main news is all the wonderful things we, as parents, have been able to do while they were at camp. They might be interested but my guess is that they are not. But given how much we are paying I feel we need to fill in the lines. So, with my son, I decided to just make stuff up. Basically, he receives an email from me every couple of days with an ever-increasing sinister plot that is unravelling. It started with the disappearance of our pet hamster (she’s fine) and then led to the disappearance of all of the pets of children who had gone to camp around Northern Toronto. Some pets have shown up at kids camps but then there was the Higgs Boson and heatwave melded into the story. Frankly, I don’t know where this is all going and now I have to extend it out for another two weeks and then bring it all to some resolution. This is a dangerous game indeed. I think this evening the hamster will reappear at home but just a little different. And of course, given that this is a one way affair I have no idea how this is being consumed at the other end. A dangerous game indeed.

Anyhow, Bunk1 don’t just rest with emails. You can also send your kids puzzles (Sudoku etc) and then you can view pictures that the Camp posts on the Bunk1 site. There are hundreds of these and, basically, what you do as a parent is sift through them to see pictures of your own children. For the first day of pictures we found our youngest and she was never smiling. This was a bit of a worry. But later on she seemed happier. Of course, this may all just be camp censorship. Who knows?

The Time article suggests parents obsess over this. We aren’t quite doing that but because it is there we do look. What is true, however, is that Bunk1 has tapped into a missing market for parent camp communication. I’m not sure I’d want a cheaper option as that might only encourage more communication. What we have here is more than enough.

Should parents tax their children for eating too much candy?

[This post originally appeared at Forbes.com on 10th July 2012]

A couple of years ago, I sat down with Chana Joffe-Walt and my then-11-year-old daughter for a long interview with the NPR podcast, Planet Money. This week Planet Money replayed the podcast and I was able to reflect on how it all turned out.

Before I get to that, here is the podcast itself.

The toilet training story was, in fact, told in my very first blog post in 2003 and there was much more to it than could be conveyed in the interview.

Now, in the podcast, the health tax accrued no revenue because it completed curbed spending on candy. Of course, we are now two years down the track with a more independent child. The issue with independence is that allows for another thing that often accompanies attempts to collect a tax: under-reporting. It did not take us too long to discover that the allowance was being used to purchase candy and my daughter owed a ton of back taxes.What was more recent was the whole notion of using an allowance combined with taxes to nudge behavior. If you haven’t a chance to listen to the podcast the story was that, while we gave our kids an allowance, we didn’t want them spending it all on candy and so imposed a steep “health tax.” The health tax was not just arbitrary but meant to be a compensation for imputed additional costs we would have to pay if they ate too much candy. To be sure, there is no calculation to actually work out those costs but it did do one important thing: it sent a signal to our children that there were costs to these actions and not all of the costs fell on them.

But this highlighted how difficult the tension between giving a child an allowance to foster independence versus providing signals as to potential adverse consequences is. The household economy will operate like the real economy. Thus, we have had to take the health issue out of our tax system and rely on more conventional forms of parenting to foster good eating habits. It also highlights that what might work — or in this case, not work — for one child can actually work for others.

Our other children don’t have the same preferences for candy and so we get some good behaviour for free. The best one can say for things like a health tax is that they put an issue on the table. They might be able to nudge behavior in a good direction but they cannot be expected to make great leaps and bounds. This is the same tension that Bloomberg will face with the ‘Big Gulp’ regulation in New York. It will be good for nudges but those who want lots of soda will get lots of soda.

So can a parent be a good manager? Sure. But can good management create perfect parenting outcomes? No.