Pixify is an iOS and Android app that I’ve developed with the help of my friend Nando Sierra (a talented system administrator), which allows you to transfer the style of an image to another one by using deep learning techniques on neural networks. From the casual user who wants to share a cool selfie to professional artists looking for inspiration or building blocks for a new art piece, a wide range of users can find it fun and useful.
You might be familiar with similar apps out there (we released our app back in July 2016, so I believe it was the third neural style transfer app available), and they all have their differences. Some are faster than others, some do video, some do stills and some do both. The one thing they all have in common, including Pixify, is the fact that they use trained models from images to use as styles. This means that on our end we run a process on each available style to generate data that the app will use to transfer the some of the style features to the given image. That is a slow process that takes hours (and not too long ago even days), and does quite some heavy lifting before hand so the user only has to wait a few seconds to see the result.
Living in an era where we all want things to be as fast as possible, it’s important to deliver on that aspect so users don’t get frustrated. I have a very particular opinion about people feeling entitled to having the world instantly owing them something they didn’t even know it existed a min earlier, but that’s just me. The fact of the matter is that if you want users, you have to give them what they want.
So here’s where Pixify aims for a different approach. It’s slower than any other similar apps out there. I know, not the best selling point, but bear with me. As I mentioned earlier, like any of the other neural style transfer apps out there, we use trained models on some of the included styles to make the process faster. We call it Presto, and it’s one of the available ‘painting modes’, as we also call them.
Presto is nearly as fast as any of the other similar apps. ‘Nearly as fast’ makes it technically slower, but that’s not what I meant earlier. Pixify has another painting mode (Custom, with some predefined configurations, or presets, of it that we call Stylized and Painting) that doesn’t use pre-trained models of the styles. This means that there’s quite more computation happening, hence it’s slower. In practice, from 20 seconds or so, to a few minutes, depending on its parameters values. Yes, it’s much slower than what the average user would hope for, but for some users the results overweight the longer waiting time.
The main benefit of not using pre-trained models is that you can use ANY image as your style. That is really cool if you ask me. Another advantage is that you can adjust a few different parameters that affect to different aspects of the process, so with a single style you can get quite different results. That’s very interesting for the curious mind who likes to play around with what’s available.
One more big plus is the result itself. Styles transferred without pre-training generate a more organic and hand made looking image. So this is the key for me, and why I really don’t mind the wait. Also, because of what I do for a living, waiting for renders to finish is something I’ve got to be used to during the years, so I’m not as impatient as most. But I’m also driven by results, and very cool and inspiring images can come out of this process.
Another thing that Pixify offers is higher resolution artwork. We’re aiming to appeal to artist and people with more artistic tendencies by offering these more ‘advanced’ options, but the thing is that they usually want to go a bit further with it, like printing them out or editing them in their computers. Neural style transfer has one caveat, it’s very demanding in terms of memory, so it has a limit on how big the resulting images can be with the current available hardware. Usually the resulting images, in any app of this kind, go just a big higher than 1k at best. In Pixify the resolution varies. In Presto mode they go from 1024 pixels to almost 1300 pixels, depending on a few factors, not controlled by the user.
In Stylized/Painting/Custom mode, they go from 512 pixels to 1024 pixels, depending on the value set by the user. The higher it is, the longer it will take, though. So, “where is the high resolution you were speaking of?”, you may ask. In Pro mode (more about that later), those resolutions will multiply by four, maintaining the same visual quality without increasing the rendering time. Now, that’s a more suitable size for editing and printing.
Pixify Pro is just a $0.99 In-App purchase to unlock a few extra features, like the high resolution artwork I just mentioned, no watermark at the bottom right of the image, longer video duration in iOS, allowing for higher values in some of the Custom parameters that have more impact in the render time, and making Magic Doodle available, a fun feature that I will let you discover by yourself 😉
One last thing to mention about Pixify and other similar apps is the way all this happens behind the scenes. You might hear all these terms, like neural networks, deep learning, etc… but beyond how cool they sound, you don’t know much about them.
For decades we’ve been trying to give machines the ability to think as humans. We’re still far from it, but a lot of things have happened in the recent years that takes us a step closer.
‘Neural style transfer’, as the technique behind Pixify and other similar apps, uses deep learning in neural networks to learn about an image and transfer those features into another one.
There are different machine learning approaches (it’s a long subject), but neural networks, a paradigm inspired by the human brain that enables the computer to learn from observational data, is especially well suited for any kind of image recognition. Deep learning is referred to a set of techniques for learning in neural networks.
The thing with all this is that, for a machine to learn nearly enough to perform these tasks, it has to observe a significant amount of data. This is a tedious task and very time consuming, and it hasn’t been until recently with the democratization of tools and techniques, availability of massive data sets and the boost in computing power, that we had the option to crunch machines with these complex algorithms to obtain results in a reasonable amount of time (from days, or even weeks, to hours).
By using pre-trained models we can rely on the device (phone, tablet) capabilities to run the process in a few seconds, but that also has a few caveats, and that’s why the few apps of this kind that offer that option don’t allow to do it for all the pre-trained styles included, but just a few (this is mostly because it would increase the size of the app significantly).
All these apps, some more than others, rely on a server to run the process. Pixify relies on a server (or servers) because we not only offer pre-trained styles (Presto), but the ability to use any image (Stylized/Painting/Custom), and that’s a process that can’t happen on the device. It needs a bit more kick.
What are the downsides of using a server? High usage demands. Well, that’s just a downside if you have limited infrastructure (see limited budget). Servers are expensive, and as simple logic suggests, the more instances you have, the more you pay. Pixify is mostly a free app that a couple of folks develop in their free time, so we can’t have just servers laying around to satisfy any kind of demand. When we have some usage spikes, the process gets a bit slower because you are waiting on a queue (and the app informs you of that).
- For artists or people with artistic curiosity
- Any image can be used as style
- Parameters to change the look of your artwork
- Four times higher resolution artwork
- More organic and hand made looking results
- Free with a $0.99 In-App purchase to unlock extra features
- Still images and videos (iOS)