Coding is for more than just apps and websites
Chatbots, IoT, AR, VR & Machine Learning— and how to learn more about them
I give a lot of talks to school and college students - about the future and also about why a career in IT is a great one. I always love showing them that coding is about more than just apps and websites, as it’s likely they aren’t aware of the numerous different things they could create.
This week I had a work experience student at my desk for a couple of hours, and I showed him a voice chatbot I’m currently working on — we rang a phone number and had a little conversation with AWS Lex, via the Connect and Lambda services. Seeing his amazement and surprise was great, which led me to write this post; all about the ‘other’ engineering you can do, outside of apps and websites, along with info on how you can learn more about them.
We may aswell start with:
Chatbots
I wouldn’t call chatbots emerging tech any more, but the recent cloud-based platforms are still relatively new and probably not used by most engineers.
I’ve made text and voice bots, neither of which were too technically difficult to create. You have considerations however you don’t have with websites such as ‘conversational design’ i.e. what is the best way to construct a conversation between a human and a software program?
When I built a chatbot a few years ago when I worked for a bank, we asked ourselves should the chatbot try to portray a “personality”? Should it have a slight sense of humour, or play it straight the whole time? Should it apologise if things go wrong? Or emphatic?
These are all interesting questions and make working on a chatbot project fun and educational.
More info:
- I’d recommend AWS Lex if you want to learn — https://aws.amazon.com/lex/ but there’s plenty of other options, including
- Microsoft — https://dev.botframework.com/ or https://www.luis.ai/home and
- IBM — https://www.ibm.com/watson/how-to-build-a-chatbot
or you could just go crazy and make your own! They’re basically if-then-else statements with some NLP thrown in …
Alexa Skills
Alexa Skills are ‘apps’ for Alexa, and are basically, also chatbots. I’ve been creating Skills for the last couple of years, on and off, and they remain a fun and easy way to build something new.
When I give tech talks, I sometimes get Alexa to introduce me, via a little Skill I made that tells the audience what I’m going to cover. It’s a nice way to make the point that they could also build experiences for Alexa, and not just be a consumer. That itself is one of the main points I love to make to students — coding in general allows you to become a creator, not just a consumer.
More info:
- I’ve written a couple of tutorials on how to make Skills, but Amazon are constantly improving the tools so you’re probably best going straight to the source: https://developer.amazon.com/en-US/alexa/alexa-skills-kit/learn
- For Google Home device: https://developers.google.com/actions/
Augmented Reality — on Mobile
Yes, these could be considered as app development, but one of the most fun and different technologies to code with is Augmented Reality (AR) — placing virtual objects into the real world, usually via a smartphone camera lens. Still also relatively new, each year more and more developers are creating AR apps and experiences, and this will only continue to grow as more devices become AR compatible and the tech gets better.
You can make games, like AR Madness, an open-source game we made in work with Apple’s AR Kit2:
or you can make portals to exotic lands …. like an office …
Recently, I’ve been experimenting with ARKit 3, which has advanced motion tracking, and is frankly, really cool!
You can also use AR for more ‘enterprise’ use-cases like safety walkthroughs of buildings, or to provide measurement tools etc. Games are way more fun though!
More info:
- I have an epic 5 part tutorial on how to build that game: here: https://medium.com/libertyit/ar-madness-our-open-source-arkit-game-tutorial-part-one-game-design-and-first-arkit-app-1ba35bea136d
- Apple’s documentation here: https://developer.apple.com/augmented-reality/
- Google’s version, AR Core: https://developers.google.com/ar/
- Unity: https://unity.com/solutions/mobile-ar
Augmented Reality — on Headsets
I believe the next big technological leap (in the consumer market) is going to be AR Glasses — smart-glasses that can do most of the things a smartphone can do, but on your face!
Imagine watching Netflix with your glasses, as you pretend to work at your desk! Or walking through one of those ‘portals’ in your living room, perhaps into a virtual shop that’s all around you. I think the possibilities are endless and will be the next developer ‘gold-rush’.
The tech however isn’t there yet — but you can still experiment with the current devices on the market, and learn about the different user experience paradigms that these devices bring such as voice and gesture control.
More info:
- Microsoft HoloLens: https://www.microsoft.com/en-us/hololens/developers
- Magic Leap: https://www.magicleap.com/creator
- And here’s a post I wrote earlier in the year, the 2019 A to Z of Augmented Reality! https://medium.com/libertyit/the-2019-a-to-z-of-augmented-reality-474f595b687a
Virtual Reality
Virtual Reality (VR) is of course another great option if you want to build something different, and is not as difficult as you may think.
We’ve used it in my work to build safety simulations and meeting places, but the big opportunity is gaming and entertainment. I’m convinced that VR Horror is the best horror you can experience — nothing can beat the feeling of there’s something behind you, when in VR there actually is something behind you!
One possible drawback to developing for VR is that the market is not as big as for mobile games, but that may actually be an advantage; the hardest part of making mobile games is getting them noticed in such a crowded market. New devices like the Oculus Quest will also drive growth and popularity, so long-term, knowing how to make VR apps/experiences should be valuable.
More info:
- Unity: https://learn.unity.com/tutorial/vr-best-practice
- Google: https://developers.google.com/vr/
- AWS: https://aws.amazon.com/sumerian/
IoT
The Internet of Things (IoT) is one of those all encompassing buzzwords that never goes away, but something that most developers have probably never dabbled in. Basically it relates to internet-connected devices; any devices. Fridges, doorbells, cameras, cows (with an IoT device on them …), whatever!
Also likely to be become more widespread in the future, they offer the chance to bring user experience out of the smartphone and laptops. I’ve experimented with the AWS IoT button — a small device with one button that can be either clicked once, double-clicked or long-pressed. When used, the button sends that up to an AWS Lambda, from which you can do anything you can think of!
That is incredibly interesting and powerful — what can you think of doing if you get a button click message to a cloud app? Amazon recently announced they are discontinuing the consumer version of the buttons — the ‘Dash’ button, used to order specific goods, but the developer version is still available and waiting for the next great use-case.
More info: https://aws.amazon.com/iotbutton/
Machine Learning & Artificial Intelligence
I have to mention the field of Machine Learning / Artificial Intelligence. Recent years has seen the ‘democratisation’ of a lot of the technology needed to use ‘AI’ or ‘ML’, mainly due to the numerous cloud services now available.
This has led to engineers like me being able to build solutions that include object classification or human recognition, relatively easily. Technologies such as TensorFlow Lite allow you to run machine-learning models on mobile browsers, bringing new capabilities to your web-apps and your users.
I’d break this whole area into two sub-areas:
- Applied AI
- Custom machine learning
Applied AI is using models and services that have already been built — like cloud-based object recognition services, whereas custom machine learning is about building your own models, based on data sets such as images, text etc.
Both can be fantastic to work with, and can open up huge opportunities for both the products and services you create, and also your own personal and career development. Every business team I talk to mentions either AI or machine learning, so it’s one of the biggest and potentially valuable new fields for engineers.
More info:
- If you want a fun way to start learning, try the AWS DeepLens — a custom built camera from Amazon that runs models locally on the device: https://aws.amazon.com/deeplens/ I have one and it’s fun!
- If you can get a hold of one of these, the AWS DeepRacer is also a particularly cool way of getting started with machine learning — https://aws.amazon.com/deepracer/
- More standard way to learn — AWS Sagemaker: https://aws.amazon.com/sagemaker/
- Google: https://developers.google.com/machine-learning/crash-course/ml-intro
- TensorFlow: https://www.tensorflow.org/
Blockchain
Only joking!
Final thoughts
There’s many other ways of using code that I haven’t mentioned here (Satellites! Cars! Drones!) but I hope this was a nice intro into something other than the usual app and web development. Any thoughts or comments please let me know!
