Steren's Blog

So much drafts, so few posts

Photographic memory

The Narrative clip is a wearable camera that automatically takes a picture every 30 seconds. One year and a half ago, I decided to back this ambitious Kickstarter project. I will try to explain why I did so and share my experience using the «clip» for a few weeks.

My Black Narrative Clip

My Black Narrative Clip

Forget it to better remember

The clip doesn’t have any button. You don’t «switch it on». It is designed to be worn and to automatically take pictures every 30 seconds. To stop, just put it in your pocket or face down on a table. This simple interaction design, its weight and size make the result something that you don’t have to care about. There is a clever “double tap” feature to take a picture and star it, but this is not the main use case. You just wear it, and forget about it.

An experiment

I’ve chosen to give it a try, mostly as an experiment. I am aware that having the ability to record anything I see may have an impact on my personal life, that’s what I want to experience.

I always considered that the most precious things on my computer were my personal pictures (today stored privately online). I like taking a few pictures of events. I have been storing and paying attention to them for the last 15 years. Sometimes, it happens that I visually go into the past and remember a moment with more details that I could do by myself thanks to these images.

Human memory

As Wikipedia describes it, our memory doesn’t store every moment of our life with the same level of details. While our sensory memory let us recall exact details of an item we just saw, we are not storing them and are most likely to forget them in the next minutes (for example, the color the shirt of the person sitting next to you, the exact amount you just paid at a restaurant…). Our short-term memory allows us to explicitly remember an information for a short period of time, but we are most likely to forget it later. We do not remember everything, we are explicitely and implicitely storing scenes of our life in our long-term memory. The fact that we remember a scene more than an other has, I think, many causes: attention, frequency, emotional level…

Photographic memory

The Narrative clip is a different kind of memory, it is lossless. It stores the same level of detail for every picture, every moment. This is, for me, a new kind of memory. It is useful? probably not a lot, but these kind of devices may change the way we are able to remember the past. It is like having an eternal eidetic visual memory, like this Enki Billal character whose quest is to remember the first days after his own birth in Le Sommeil du monstre.

I think being able to go back in time is priceless. Being able to live a second time a given day you experienced a long time ago is an experience that is not possible today, but that we may see tomorrow, and Narrative goes into this direction.

Technically, the pictures taken by the clip are not ideal, the camera’s field of view (70°) is really small compared to the human eye (or a GoPro) and the image sensor could really be improved for dim light. This often leads to noisy pictures, not capturing the whole scene and of a strange framing. While these are negative points and could definitively be improved in future versions, they are also a way to be later surprised of what was extracted from a moment. And the real value is in the series of pictures, that together, create a moment. Most of the time, I am not focusing on one particular taken image, I prefer reviving a moment in a time-lapse mode with the Narrative application.

Wearing it

After a few weeks of wearing the clip when I felt confortable doing it, I have more answers to my initials questions: How will people react? Will people or I behave differently? When will I wear it?

First thing, most people that are not in direct interaction with me simply don’t notice it. I think it’s because it’s very discreet, and nobody knows (yet) that these kind of device exist. People I talk directly with notice, or I try to present them the device quite early when we meet. Most of them are curious, and once I explained the principle, they don’t mind if I continue wearing it and very often ask me to send them a “best-of” of the event. I was at first wearing it all the time except in professional situations, it continued but I realise now I sometimes forget it when performing routine tasks.

A lot of data

One pictures from the Narrative clip sizes 2048×1536 and weighs around 250KB. I notice I gather around 750Mo of data per day, which are automatically uploaded and stored encrypted, after analysis, on Narrative’s servers.

What’s next

What’s next is that by storing so much information, we will need powerful personal search engines. Not only being able to search by date in a efficient UI, but also by location, by action (sitting, walking, driving…), by people (face recognition), by image elements (food, grass…). The Narrative apps are far from this and I hope they will improve their mobile apps and offer a great desktop web portal over time. But maybe it’s not their job and should just open a clean API to let other people and companies organise the content.

This powerful search is a vision, but we are not that far from it when we see the power of Google+ personal photo search for example.

Not only pictures?

This idea  goes into the broader trend of lifelogging. I already passively record my position with Google Location History, the music I listen on last.fm, log a lot of things on Foursquare and Evernote… which tagline is, by the way, «Remember Everything». That says a lot on their vision, it could have been “The best app to take notes“, but no, they see larger than this, they want to help you remember, and first step into this direction is to help you take notes.

Except for a few notes, check-ins and pictures, gathering all this data is pretty useless today. But I am confident it may have more value in the future. It will be raw resource that other services will tap to generate real customised and personal value. And this value will certainly be more than editing a movie of your life after your death like Robin Williams is doing in The Final Cut.

The Technological Guide to a Foreign Language

Two month ago, I moved to Germany, and unfortunately I did not speak a word of German. Here is a set of tools I have been using to be able to find my way in this foreign language:

Browsing the web: Chrome

Of course, Chrome (desktop and mobile) has a built-in translation feature. When navigating to a German website, an instant translation of the entire website seamlessly replaces the original one. I find the user experience quite good, because seemless.

Chrome's translation bar

Chrome’s translation bar

Instant translation in the street: Word Lens

You stumble upon a sign in the street, and have no clue about what it is saying. First thing that comes in mind is to open the Google Translate application on your mobile, it even does not require an internet connection thanks to the offline download of language packages. Unfortunately this would require re-typing the text in the input field (the picture input does not work offline, and has poor results).

The solution I am using for quick and dirty translations is Word Lens. If you never heard of it, it is an augmented reality app that replaces real-world texts by its translation. Here is the promotional video and a real-life usage example:

left: real world, right: same scene through Word Lens

left: real world, right: same scene through Word Lens

It is a great combination of interesting technologies: real time OCR, translation and image manipulation (to incorporate the final result on the original image). It gives instant result in its more natural way, which is a great user experience. Unfortunately, for its price, I think that its stability and user interface could be improved.

I’m really looking forward to try this using Google Glass, the great news is that apparently an  app is already available. It will remove the need to pull out the phone and launch the app, however, is will not be full augmented reality as Google Glass is only a Head-up display.

Translating paper documents: OCR

The worst is when the data is not available digitally, for example, received official documents. Translating them word by word or manually re-typing them were not options when you receive a 10 page health insurance contracts. Once again, free online Google tools can help with this task:

    1. Scan: Unfortunately, a scanner is needed to get a PDF file containing black and white images of your document. A simple “portable scanner” can be very handy, but ideally a multi-page scanner is even faster. In my case, I wait the next day to scan them at the office.
    2. OCR: Drag and drop the scanned PDF to the Google Drive website, make sure to enable the conversion to the Google Docs format on import. This will create a Google Doc document containing images of the scanned pages and the automatically extracted text in its original language.

upload

    1. Translate: The final step is to translate the text, no copy paste needed, Google can translate a full text document to a new one, keeping its layout and style.

translate

  1. Profit: You end up with the entire document translated from German to English. Below each original scanned image you find the translated text with a similar formatting. And if like me you prefer to archive digitally your papers, the great side effect is that you have scanned your paper.

Learning: Duolingo

Of course, nothing will replace to actually know the language. While old-school lessons and grammar books are very often the best strategy, I have to mention duolingo, a really great application for progressive and regular lessons

The german skill tree of the Android duolingo app

The german skill tree of the Android duolingo app

Duolingo is a free mobile and desktop application will help you learn a new language using game mechanisms : your goal is to increase your level, you earn points each time you complete a lesson, you have to unlock current skill packages before going further…

The learning curve is really well designed so that you are making progress each time you use the app. The fact that it is a mobile application and lessons are short let you use in your daily commute for example. It is very well though to keep you learning  the language across days: You get reminders, you can compare to your friends, using it everyday increase your earned points… And the app itself is so much well designed: its graphics are very nice and user interface is really well tested.

What’s interesting is that behind the scene, duolingo aims at using crowdsourcing mechanism to provide translations to customers. I  wrote about this mechanism in a previous blog post.

Don’t worry, I’m not restricting myself to these technological tools  to learn German, but I thought they were worth listing to emphasise how modern technologies can help us in what we would have consider a very difficult environment before. And you, do you have any other technological tip?

Auf Wiedersehen !

Salvador Dalí’s tiger, quite inspired by an ad for Ringling Bros

Dali Ringling Bros tiger inspiration

Left: Ad for Ringling Bros. and Barnum & Bailey, Charles Livingston Bull 1915

Right: Dream Caused by the Flight of a Bee Around a Pomegranate a Second Before Awakening, Salvador Dalí 1944

A poster I found in my attic reminded me one of my favorite piece by Salvador Dali, I think we can agree one got inspired by the other.

Cocktail dosage in martini glasses: Maths are here to help

New Year’s Eve is getting close, you’d beter be prepared. Let me share a problem I had last day when I decided to make cocktails in martini glasses. How was I suppose to dose my ingredients in such cone-shaped glasses?

cocktail in a cone-shaped glass

How to properly dose a cocktail in a cone-shaped glass?

Let’s take a first example, The Black Russian recipe is:

2/3 Vodka

1/3  Kahlúa

The problem is that you cannot apply these proportions to the heights of each liquid since the glass is not a cylinder. For a same height, the volume will not be the same, and your cocktail a failure.

Our question is:

What is the ratio between the heights of the two liquids that will make their volumes meet the recipe proportions?

What we have is two volumes vV and vK and their proportion to the total of liquid Vtotal = vV + vK:

And what we want is the ratio hV / Htotal between the height of Vodka and the total height of liquid.

cone

Volume of a cone is ,   being the base with its radius r.

We then use Thales’ theorem to link any radius r to its corresponding height h:

By using the two last formulas in our volume ratio equation, we get:

What we demonstrated here is a kind of Thales’ theorem (Intercept theorem), but for 3D cones. We obviously have the same kind of result with surfaces of triangles. That’s interesting, that’s a result I never encountered in the past. And it doesn’t only apply to right circular cones, but to any type of cone (pyramids for example), this can be proved by computing the base B using an integral.

The answer to our initial question is then:

That’s right, to have 2/3 of vodka in volume, the height of vodka should be almost 90% of the total height. If the ratio had been 1/2, the height of the first liquid should have been 80% of the total height. It’s not that obvious to me.

Last point, if your cocktail has more than two ingredients (for example a White Russian), I would suggest to always consider cones and not bother with substractions:

1/3 Vodka

1/3  Kahlúa

1/3 Cream

Start by adding 1/3 Kahlúa (70% of total height), then add Vodka until 2/3 of volume (90% of total height) and complete with cream.

Enjoy your cocktails, and happy new year!

Noisebox: a connected music box

A friend came up with a brilliant hack during Hack Day Paris: the Noisebox. Last Christmas, I offered one to my girlfriend, watch the video:

Today, like many other, I listen my music digitally, I forgot about albums and cd players. And thanks to streaming services I can listen instantly to every track I want from any device. That’s great. But like me, don’t you miss the sensation of having a small physical collection that represents something for you? that you picked, that your friend can look at when they visit, that you can re-discover from time to time, that is truly yours ? CDs are just here for technical reasons. Let’s just stay with the idea of objects. That’s what the box is all about. It simply plays music associated to objects. And more than CDs, these objects can really be unique, personal and sentimental.

Pierre build the hardware and I developed the server using NodeJS and MongoDB. I hope we will find time to publish more documentation and do a proper release of the project.

The Noisebox and its associated objects

Hack Days

I realize I never wrote about it: In 2011, I’ve been involved in the organisation of two “hacking” events (hackathons). I will describe them and try to give some feedback about the organisation of such events.

The main goal of theses events was to gather around the spirit of Hack people who can do things (developers, designers, makers…). I’m convinced that the meaning of “hack” is misunderstood, especially in France where it’s always associated with piracy. We tend to forget that many innovations or great companies of today come from little disruptive things hacked in a garage. For these hack days, the idea is that in the end, participants have to demo their hack, no slides are accepted, only prototypes!

Hack Le Camping

(June 24-25 2011)

logo for Hack Le Camping, designed by myself.

This first one came from an idea we had at Le Camping, the parisian startup accelerator were I built Beansight. I took the lead in the organization.

One of the goal was to let the startups from Le Camping open their APIs for the first time to developers.

The event was located at Le Camping, which is usually a place where 12 startups work everyday. So the place was not an issue and could welcome 80 guests.

With the help of others, we gathered two main sponsors. The deal was to let them promote their services before the event and be present on the printed materials, in exchange of €1000 each. This budget covered the food (3 meals) and also allowed us to print t-shirts for every participant. Because we didn’t find a great prize that could fit either a single hacker or a whole team, we decided to simply offer a prize of €500.

A hundred people registered to the event, and in the end around 40 hackers showed up for the event. I did not expect such a large difference, I think it’s due to the fact that the event was free of charge, a lot of people registered and never showed up later.

Participants hacking in Le Camping

The event went very well, the 24 hours were very intense for the participants. We added soon after the kick-off a stand up session for every project to explain what they were working on to the others.
In the end 10 projects were demoed. The overall winner was a team from Moodstocks. They created a fast-paced scavenger hunt mixing real time technologies from Dabla and image recognition: a player selects an object to look for, the first from the others to scan it with his phones takes the lead.

If you want to learn more about the event, here are the schedule of the event, the slides of my intro talk and a list of the hacks.

Hack Day Paris

(November 4-6, 2011)

Hack Day Paris logo, designed by Simon from Joshfire.

Soon after, I met with Skylar and Sylvain and we started discussing about a larger event.

The vision was to organize a large independent hack day in Paris, something that had never been done before. We targeted 150 hackers and wanted to gather a jury of renowned and respectful people. We did not wanted to impose a theme, but preferred to reward the “most brilliant” hack. We wanted people to work on what they liked, to produce something awesome.

We soon decided that tickets would be €10, not to make money from it, but to make sure that people registering were people really willing to come. In the end, around 140 hackers showed up. I think our strategy worked fine.

At first our main concerns were the place and communication. People from the co-working space La Ruche liked the project and its philosophy, so more than a place, they joined the project and helped us organize the event. We never could have done it without them. Communication was done by reaching all our networks (Paris Hackers, the startup ecosystem, schools…)

We gathered around €9 000 from sponsors and entries. This allowed us to provide quality food for everyone all along the 40 hours. We printed badges and rented additional Wifi.

Around a month before the event, a lot of effort had to be put into logistics: we had to make sure every people involved agreed on the schedule and the use of the places. Because many were involved, this was not an easy task at all.

The event went very well, as usual, as an organizer, you spend your time handling little issues here and there, making sure everything runs smoothly for the participants. We did shifts so that at any time during the 40 hours, one of the 3 organizers was awoken.

Demo time, in Le Comptoir Général, a bar next to La Ruche.

We had great feedback concerning the  location: La Ruche is not a regular place for such event, it’s made of little rooms, with outstanding decorations. This had a great impact on the spirit of the event.

Around 40 projects were demoed at the end the sunday afternoon. Here are some remarquable examples: a real time chess game (Ninja Chess), a place to give old things (Freesbee), a musical keyboard using balloons , a wine suggestion service (Wine Combinator)…
Once again, the grand prize was €1500 for the best team. Side prizes were organized by sponsors. The winners are a team who built a music box that links objects to music in the cloud. I will blog about it later.

For more info, visit the official website, have a look to the pictures of the event on the facebook page.

For sure, I will be involed in other hack days in the future. And I hope this article can help the ones of you who want to launch such events, feel free to contact me.

Use Google “search by image” to track your pictures (or to spot fake facebook profiles)

Google released two months ago an update to its image search engine: You can now search by providing an image as input.

I see some use case for this feature, a powerful one is to allow creators to track their pictures.

Track your pictures
Let’s take an example, I created this wallpaper 3 years ago:

Creative Commons BBB

A wallpaper using characters from Big Buck Bunny

I released it under a Creative Commons Attribution license and put it on flickr, so people were welcomed to use it.

I was pleased to see that it was used on hundreds of webpages (including WikiHow), even in languages I don’t understand like polish or finish, and most of them are giving attribution.

Similarly, a clipart of mine (released in the public domain) was used for various applications: create custom plates, make fan art of Alice in Wonderland, sell caligraphy, or cupcakes and promote a church.
Many of my wikipedia image contributions are also findable across the internet (for example to write about dinosaurs or to portrait Christophe Salengro).

Copyright infringement

Let’s take a well known copyrighted image, such as this portrait:

Google similar image for "Afghan Girl", copyright Steve McCurry

Well, seing all the Google Image results, National Geographic could easily annoy the website’s owners by asking them to remove it. That would be crazy? Well consider that this kind of removals are systematic on Youtube, and, as Eric Schmidt said at the eG8 summit in Paris, could be automated and applied to the whole internet (they have the technology, it’s called Google Search by image).

For sure, this new little feature can be very useful for photographers and artists who want to protect their creations by tracking who is illegally using them. Visually searching the entire web has never been possible before, now it’s as easy as a google search.

But let’s finish this article on a funnier note:

Facebook fakes

Friends of mine were arguing with a girl on facebook and they suspected her to be an impostor.

a suspicious facebook profile

A quick Google search by image showed us that this picture was called “Portrait of happy young lady smiling” and sold on sites like fotolia. Some minutes after briging this proof, he/she deleted his/her account. I love technology.

Bokeh

I’m not a huge fan of the Black Eyed Peas (I even think David Guetta killed the group), but while watching at the “Just Can’t Get Enough” video clip, I enjoyed the work on light and noticed for about one second (at 0:54) a very subtle shot:

This beautiful blur is called a bokeh. It appears when an out-of-focus point of light is captured by the camera. Some photographers like to play with it.

Usually, the light spots make discs on the picture, but here they are hearts. How come ?
Well, the shape of a bokeh is linked to the shape of the aperture of the camera. So I bet the photographer used an heart-shaped aperture in this shot.

Of course, this effect could be computer generated, a convolution between a heart and the image does the trick, but let’s believe it’s not a fake.

my camera and hand-carved business cards

I wanted to try it at home to see if anyone can do it, it appears that it’s pretty straightforward: just carve a heart in a card and put it in front of the camera. Then point some lights and manually set them out-of-focus. Results are pretty good.

Bokeh with an usual aperture

Same scene with a heart-shaped aperture

re-Captcha, nanojobs and GWAP

Probably the most clever idea I ever heard of.

This is not new but it keeps astonishing people when I tell them about it: Did you know that Captchas help scanning books? And they are doing it very very well.

a captcha seen on the facebook registration page

You all know about captchas: Images containing words that you are forced to type to make sure you are a human and not a robot when performing various actions on the Internet (create an account, write a comment…) in order to fight spam. Thousands of people are decoding them everyday. Everyone of them is doing a small mental effort to read the words.

a captcha from re-Captcha

And this is where the guys from the re-Captcha project had a brilliant idea: What if these thousands mental efforts could be used to actually do something useful? Like helping scanning books ?

Today, many organizations are scanning old books and transforming them in a digital format. The OCR software that transforms the scanned image to digital text may sometimes not be able to do its job correctly (however complex the software may be). re-Captcha uses the human brain to actually decode words the computer is not sure about:

  • What you see when you look at a re-Captcha are two words.
  • Among those two words, one is known by the server. The other one is a word the computer knows it didn’t managed to read properly.
  • You being a human or not will be judged on the first word, and the result you enter for the second word will be used to decode it.
  • Then we can imagine that if a certain number of users read the same for a given unknown word, it is most likely to be the right translation.
  • By using this technique, the system is able to digitalize texts with a higher precision that most of the other OCR systems.

The company behind re-Captcha, its data and its market-share were acquired by Google in 2009. (what else would you have imagined)

To me this system is brilliant: it solves a problem by dividing it in such simple tasks that they can be executed by people who don’t even notice that they are working. (And what’s nicer with this one is that it helps fighting spam and digitalizing books, two great causes.)

nano jobs
I don’t know if there is another term, but I call this nano jobs.

Let us take another example of nano job: in 2006 a professor released a fun tool where you can play with a random other player: an image was displayed and your goal was to find common words describing this image with the other remote player. Of course, you quickly realize that this was only done to help labeling the image base: Today, contrary to a human, a machine has difficulties to understand what an image represents (Image recognition). The “find common words to improve your score” is just a incentive to gather a lot of data. Google did the same to help labelling its image base.

Playing the ESP game with a random player, finding common labels for a given image.

This leads us to another important point in nano jobs: game mechanics.

You cannot force people into doing small tasks, they have to do them by themselves. In the case of re-Captcha, they understand the need of fighting spam, so they accept the task. In the case of the ESP game they want to do the best score or maybe to have fun with a random web user (this reminds me chatroulette).

These games are called games with a purpose (GWAP). Imagine, the workforce that these millions of people farming like zombies on Farmville represent (Unfortunately, Farmville business model is more in selling your data and selling stupid virtual stuff than making you doing nano jobs). Then, when we hear about Google investing in social game companies, I think nano jobs are part of their motivation (not the only one of course).

My conclusion
To conclude, I think this de-centralized and effortless way of solving problems is extremely powerful. Once again, divide and conquer seams to be the strategy to adopt, even for problems that don’t seam scalable.

Some more examples

  • gwap.com seams to specialize in Games With A Purpose. You can play to help tagging images, music, find synonyms, image tracing, emotions from images, image similarity and video labeling.
  • GOOG-411: Google opened a vocal search phone service to provide search results by phone. It seams that the goal was for Google to gather a lot of voice records to improve their speech recognition engine. (to provide data for machine learning)
  • On a similar way, Picasa performs face recognition, but it’s not perfect and you have to help it tagging your family and friends in your pictures. Well, the more you help it, the more it will be accurate later, and on a larger scale, the more Google is gathering learning data.
  • Google, once again, provides a free Translator toolkit to help doing sentence to sentence translation. This tool is free, but what you may not know is that I bet we are feeding Google with translation data by using this tool.

To another extent, Amazon is providing an online service called Amazon Mechanical Turk. It links nano-jobs providers with a widespread user base doing small tasks for money. I heard many companies are using this platform to help performing Human Intelligence Tasks.

Phone books privacy

Let me remind you that anyone can freely find your address and phone number in phone books, and it has been possible for decades, at least in France.

So please, stop complaining about Facebook breaking into your privacy when it allowed apps to ask you to give them your phone number and address. If you want to fix your privacy leaks, start from the basics and remove yourself from phone books. Personally, I only blame Facebook for not making this sharing disclaimer clear enough.

Follow

Get every new post delivered to your Inbox.