[From our friends] Dicify AI
Making art from science using AI in TensorFlow.js
Did you know you can draw using Legos, Post-it notes, wood chips, and even playing dice? All you need is a lifetime of artistic skill… or… a fun project in TensorFlow.js for the capstone of your book!
The idea was simple: teach an AI to draw with dice and provide me with a “color by number” solution so I could have an assured victory. I know it’s not terribly difficult to eye up a design and build it, but that would have taken me forever, and to be honest, I can’t even get the spacing right when I sign a birthday card 🤦♂️.
Because I know my limits, I decided to start with something easy. My first endeavor is to recreate our company logo with dice. The logo for Infinite Red, Inc is pretty simple and should work fine with a few dice.
Now I won’t bore you with the details, so we teach an AI to cut an image into patches and to identify the die that works best for each patch.
Not bad, but it requires quite a few dice to get definition, and the darkest you can get is a six, unless we decided to allow multiple colors of dice. If we can add inverted dice, we can effectively get double the usable patterns.
With some extra refinement on the data, you can train the model to handle two different colors of dice.
Because the code was in JavaScript, it wasn’t hard at all to overlay images and make an updater that adds new data points to the training data, while helping you refine images.
Now that TensorFlow.js is in place and I’m able to touch up my data as well as the finished product, I can attempt my first work of art. Again, I didn’t want to go too ambitious, so I decided I’d just fill a 12x12 frame with dice. By my calculations, I’d only have 19x19 (361) dice to work with. This doesn’t provide the clearest visual, but I feel the logo is still recognizable.
To solve this design I need around 400 dice. I was able to find decent prices for common white and red dice on Amazon. The sum total for this project was around $50 of dice and a $10 shadowbox frame.
Once everything arrived, I printed out the dice and got everything ready on the table. I have to admit, it looked really cool to have so many dice.
I started by using the shadow box as a trough and placed dice safely inside. It was easy! When I was about halfway through, I realized I was building the design with the glass on the back… so I’d have to flip the whole thing. Whoops!
I successfully flipped the design and finished the pattern with all the dice and image reversed. I loved the backward version! It looked like it said “GO AI”. This requires fancy fun lights for an animated GIF.
Now that the pattern is reversed, the only way I could check it was to place it on my glass table and lay on the ground, and stare up through the glass. When I did this, I noticed the red dice were awesome and translucent!
Once I noticed that I hooked up the lights behind the IR and made the final product glow. I really love it!
I shipped this amazing work off to our CEO, who promptly opened the gift, disassembled it, and immediately began playing craps with his real friends. Just kidding, he thanked me. He really appreciates hand-made gifts.
Now, I’m ready to push myself a little further. Can we do more dice!?!?!
I dare say, the 12x12 inch was quite heavy and the resolution was minimal for a minimal design. Since the AI was programmed in TensorFlow.js, I figured it would be quite meta to re-create the logo in dice.
This logo will need a lot more definition. How can I pull this off? ALAS! Micro-dice! 5mm should be great for resolution and size.
Just look at this beautiful pattern that fits in an even smaller frame.
Small dice are cheaper, but more small dice are needed for this one.
With nearly 5x the dice of the first design and lessons learned, the design was laid reversed. I also immediately learned that micro-dice were engineered in a government lab to destroy sanity. With the slightest nudge, 5mm dice will pop all over the place like ping-pong balls. Additionally, the 5mm black dice are ever so slightly smaller than the 5mm white dice, which causes some very strange complications.
The piece took about 10 hours to complete. The result was a beautiful 8x10in black frame with black and white dice.
I mailed this one off to the awesome Senior Developer Relations Engineer for TensorFlow.js at Google so that he might have a tangible work of art to show devs who are interested in getting started in AI.
Are you excited to see what kind of amazing things you can build? Be sure to check the #MadeWithTFJS hashtag and watch Jason’s amazing TensorFlow.js “Show & Tell” episodes on YouTube for inspiring demos.
If you’re completely new to AI and Machine Learning, but you’re proficient in JavaScript, I highly recommend you join me for a small O’Reilly adventure.
By the end of the book, you’ll understand the key aspects of what you’ve seen in this blog post, and who knows? Maybe you’ll come up with something even better. You can always let me know what you come up with, and at the very least, you should follow me on Twitter because I’ve always got a new adventure lurking.
In Closing
I’ve really enjoyed the results of this extremely simple network. You can easily get by with a single die color if your picture is big enough. And I’d be happy to share the dataset (it’s relatively small) with anyone who’d be willing to share their results/suggestions. I’ve seen similar works where people are using cuts of wood to recompose art with parquetry.
I’m not sure of the best way to train/recompose dice to make a drawing, but what a grand experiment! I might even upload the current dataset to Kaggle. Additional strategies I wouldn’t mind trying:
- Auto-encoders?
- Changing the input to include neighboring blocks for context?
- Style transfer?
- Perhaps even a CNN?
There are all kinds of ways one could apply the benefits of machine learning. If this isn’t what AI was made for, then I just don’t know what is.
Fun is only half of what’s important. If you or your company needs AI in JavaScript consulting, don’t forget to reach out to us at Infinite Red. I’ll be ecstatic to work with you and your team, provided you don’t own any micro-dice.
Interested in learning how to bring AI and Machine Learning to the web with TensorFlow.js?
JavaScript lets you create front-end websites that can leverage the power of AI directly on the browser. Learn from scratch with this book.
Gant Laborde is a co-owner and Chief Innovation Officer at Infinite Red, published author, adjunct professor, worldwide public speaker, and mad scientist in training. Clap/follow/tweet or visit him at a conference.
