Machine Learning mastery made simple

Machine Learning mastery made simple

Josh: It's pre-trained to detect objects that look like building footprints in imagery, and you just point the detect objects tool at that model and your data, and it will go away and create features.

Disclaimer: This podcast is brought to you by the team at Esri Australia. To get your hands on more short, sharp, and immediately usable resources, head to the Esri Australia website, and search for goldmine.

Wayne: Welcome to GIS Directions, I'm Wayne Lee-Archer.

Josh: I'm Josh Venman.

Ta: And I'm Ta Taneka.

Wayne: We're going to take a trip down the fast and happy path to success with AI and deep learning in ArcGIS.

Ta: I like the sound of this one. Now my first foray into deep learning was neither of those. So I'm sure you'd agree, Josh.

Josh: Yeah, look indeed. We shared the same pain a year or so ago didn't we, when we were preparing for Ozri, but we got there, we got there in the end. But today we can hopefully set folks off down a much smoother path than we followed to glory.

Wayne: It doesn't have to be that hard and it is much easier these days. All right let's set the scene a bit with a little bit of background. I'm going to start with one of my favorite electronic music bands, it's a Canberra group called, um, B(if)tek and the opening track to their album 2020 was called "machines work". And in it there's a sound clip that claims, machines can do the work so that people have time to think.

Now this clip was from a 1967 TV advert for IBM, which was made by Jim Henson, the puppet master himself. So, what's the point of this little side note? Well, it's the idea that machines can be taught to do our boring tasks and it's by no means new. So,  for over 50 years now, we've been revisiting and revisiting this idea with mixed results until fairly recently.

And today we've come a long way, especially in the field of machine learning or AI as it's often called. And now we have some really, really great tools that are ready to use out of the box without having to be a machine learning expert.

Josh: That sounds great, Wayne, but just saying, I don't think your musical taste is well, mainstream?

Wayne: Oh, here we go!

Ta: Shots fired!

Josh: Anyway. Carry on.

Ta: Now look, I do want to say that some of these things that we're talking about just wouldn't be achievable if you had someone sit in front of a PC with a bunch of data and you let them lose.

Wayne: Yeah.

Ta: You know, it might take too long. Could it be too complex? Is it too much scope for human error in missing things or being inconsistent?

Josh: So, we're talking about ways to make use of deep learning and use it in a simple and fast manner. But before we do that, let's look at the three things that kind of make up a typical, deep learning analysis. And to do that, let's focus on a simple example of say, looking for solar panels on roofs in imagery. That's pretty, like pretty easy example.

So I'll kick it off with number one. First of all you've got to get data, lots of it, because you've got to be able to train your deep learning model with plenty of examples of what a solar panel looks like. The more, the better.

Wayne: Oh, It's true. And it generally does, I f you're training a model up from scratch, it does take a lot of data, you're absolutely right.

So, number two, after you've done that, you take that data and you train an object detection deep learning model, and you show it all of those examples of solar panels, some of them tweaked, you tweak some parameters of the machine learning model. Give it some real beefy, gutsy computing resources. Plenty of time, go make yourself a coffee.

Wayne: And you let the model figure out an optimal approach to take, to finding solar panels. I have to say also that it takes a bit of luck and a bit of a leap of faith, because most of us mere mortals really just don't get how these models work underneath the hood and what all of the maths that goes into it is.

So, if you're happy to treat them as a black box, then they can be really, really useful and reusable, which we'll discuss in a little while.

Ta: Absolutely. And now finally, the third step here is to take those trained object detection models, show it some imagery and have it pick out the solar panels, and then generate some GIS features for you.

Now, at this point, you could use lots of different images of urban areas and then use that same trained model to go hunt down the solar panels in them. And that's really the key here. That once it's trained, it's learned and you can use that model again and again, and again, on similar imagery.

Josh: and, it sounds good right, but having gone through all those three steps myself with Ta, it's no secret that the first two as Wayne pointed out can be complex, time consuming and you can easily end up just kind of moving forward on a wing and a prayer.

But what if you could jump straight to step three and just start using it and all the hard stuff's done for you? Sound good?

Wayne: Oh, that sounds awesome, doesn't it?

Ta: Yeah, I've signed up.

Wayne: Good news, There's good news. This is where we're heading today. What we're going to be talking about is a place where you can do just that. Esri have started building out a collection of pre-trained deep learning models already, and they've made them available up in the ArcGIS Living Atlas. That's hosted up as items up on ArcGIS Online.

Now you've heard of tile packages, scene packages, geo-processing packages. Well, there's a new kind of package, it's a deep learning package or a DLPK file. And you can either use them from directly in Tools in the deep learning tool set in Pro, but also if you want to take advantage of heavier duty infrastructure, you're also able to use them in Image Server in your Enterprise set-up.

Ta: There's currently 20 different models in that collection. And of course, we'll include the link to where you can download them in the resources for this episode.

Now, whether you're dealing with imagery or video or point clouds, there's plenty to choose from. Now to give you a taste of what you can find, I guess let's share our top picks. Josh, do you want to kick off?

Josh: Sure. So, I'm going to stay on theme and revisit our shared deep learning pain Ta for one more time.

Ta: That memory.

Wayne: You guys have been scarred, haven't you?

Josh: No, It's all good. And I'm gonna highlight one of those 20 models called ‘Building Footprint Extraction USA’. And I'll come back to that USA thing in a minute, but this makes the whole process of looking for the outlines of buildings and imagery so much simpler than when we did it. It's pre-trained to detect objects that look like building footprints in imagery, and you just point the detect objects tool in Pro, Ready steady Pro, Ta.

Add that model and your data, and it will go away and create features. And it's just so simple to use. You still need a decent computer to do that on. What made me stop and think about this one particularly was A) the amount of effort it took to train a model to do that myself. But also, what was interesting is that inclusion of USA on the end of the title of the model.

And I know…

Wayne: Yeah, what’s with that?

Josh: Yeah, there's also one in the collection called ‘Building Footprint, Extraction Africa’.

Wayne: Interesting.

Ta: Shout out Africa.

Josh: Shout out to Africa, but clearly, something that's good at spotting a building outline in a typical urban US environment. It's a very different story in Africa. And it highlights the fact that there is no universal model, but that one, my pick very useful.

Ta: Very cool.

Wayne: My pick is the new road extraction model. I think this is super sexy. Really cool. Now this model is used to extract road networks from satellite imagery. So, like Josh said, you, you grab the model. You do need a little bit of beefy computing power. You bring it down into Pro or into Image Server.

You point it at some satellite imagery. And what comes out of this, is a fully connected routable road network.

Ta: Oh, wow.

Wayne: How cool, how cool.

Josh: I can see a real use case for that using Navigator and being able to bring together the existing kind of Esri routable network, plus the bits that you've discovered that are part of your world. Bring them together and then find your way using Navigator.

Wayne: Yeah.

Josh: Good one.

Ta: Very cool. Well, for my pick, I couldn't decide so I have two. Is that okay?

Wayne: Ah, why not.

Josh: Okay. Go on.

Ta: Thank you. For my first pick I'm going to veer off the 2D imagery-based models here and talk about object tracker, a deep learning model for tracking objects in motion imagery.

Now, you mentioned before Wayne, just about transport networks and Josh you mentioned using Navigator. Now this is a great tool for transport and traffic navigation, where you can track the location of your fleet or ensure they take the route most suitable to their load.

We've had a number of end users that we've worked directly with, who are using Navigator, and they want to ensure that their fleet drivers are taking routes that they recommend, particularly for height and weight restrictions.

So, using the object tracker would be an absolute game changer for all of those transportation and fleet networks.

Wayne: Good call.

Ta: Now I mentioned two here. So, my bonus model is crowd counting. It's a deep learning model, to count the number of people in an image. I know we were talking recently just about the amount of work that we've been doing and just looking at, you know, what's happening with this pandemic at the moment. And I think this is an extremely relevant model.

And it's a useful tool for law enforcement and public health and emergency response services, you know, for planning and enabling these public service providers to ensure that, you know, we're all safe, so it ensures safety and wellbeing and safe tracking of individuals.

So use this crowd counting to take a look at pictures that are uploaded either at events or wherever they've been shared publicly. And then if we need to do like contact tracing or community outreach, we can definitely use this model.

Wayne: We'll make sure that people are following the four-square meter rule. So, you can count the number of people in a location and make sure that COVID compliance is happening, so important right now.

Ta: Absolutely. And I'm all about the people, you know, whatever works for everyone. Got to keep everyone safe. Stay Safe!

Josh: No, that's, that's a three apps, well, four absolute crackers. So, any tips on how to be successful in this other than pointing them at those models?

Ta: Yes. From personal experience, I'd love to jump in here and say, start small.

That's my key learning from doing this myself. So don't unleash, a building footprint detection and analysis on imagery for a massive geographical area. Try it on a much tighter extent where you can quickly see results and get a sense of whether it's going to work well for you. So definitely start small.

Wayne: I'm going to get my geek on on this one and hopefully sort of hark back to what you were saying about the USA versus Africa notion there, Josh, and my tip is about taking these models further by retraining them.

So, what we've discovered with deep learning models is, you can get a very well-trained model, you can snap off the last layer of it, show it some more of your specialist data, and it will learn just the little extra bits that it needs to learn, to be more specific and more accurate at recognising things in your data.

So, you can take a generic object detection model and retrain it to be very, very accurate at being able to pluck out, say different aircraft and actually specifying which aircraft it can see in a satellite image of an airport.

This is really cool. And you can do this in Python Notebooks, and there's a whole heap of online resources to learn how you can retrain existing models to be more specific to your own use cases. So, I'll put those links up on the, on the show notes as well.

Josh: Cool. Start with the bought one, one that you know works and then make it your own.

Wayne: Exactly.

Josh: Final one from me, kind of a tip, but a tip for the future and not too distant future, but coming soon, you'll be able to consume these models from the Living Atlas directly in ArcGIS Online Imagery and them against your own uploaded imagery.

Wayne: Cool.

Josh: So that opens up the scenario of doing all of this without any need for ArcGIS Enterprise deployed or any infrastructure.

Ta: That's great.

Wayne: Machine Learning in the cloud.

Ta: That makes it more accessible to everyone. That's fantastic.

Wayne: All right. Now, one more. If you're going to be trying these out, you're probably going to need to go and download the Deep Learning framework libraries for ArcGIS. Now they're up on GitHub as well, and we'll make sure that there's a link to those in the website.

Hopefully with the help of these, you will have machines working for you so that you've got time to think.

Ta: Well, we did end up at a deep learning happy place, and I hope that it's given everyone the confidence to go explore this themselves. Now to help put these tips into action, we've added all the resources we've spoken about to our website, that's gisdirectionspodcast.com.au.

And we'd love to hear any tips from you guys. So, jump onto the website to send them through or connect with us through Twitter or LinkedIn. Send us DMs guys. We love to hear from you.

Wayne: Thanks for joining us. Stay spatial.

Josh: Until next time.

Ta: Happy mapping.