What's up with the weather

What's up with the weather

Grab: It's a bit like a cube of data where you can have multiple measurements that exist over time. You can slice and dice and do analysis of various different weather attributes across time and across spatial regions.

Disclaimer: This podcast is brought to you by the team at Esri Australia. To get your hands on more short, sharp and immediately usable resources, head to the Esri Australia website and search for Goldmine.

Wayne: Welcome to GIS Directions. I'm Wayne Lee-Archer

Ta: And I'm Ta Taneka.

Wayne: And we're kicking off season three of the series by welcoming back Simon Jackson, a previous guest to the podcast this year. Good to have you here, Simon.

Simon: Thanks very much for having me.

Ta: It's really great to have you back Simon. And we should let our audience know that our co-host and regular voice of reason Josh has moved on to new pastures. So expect to hear more from Simon and also a brand new host in the coming series. So please do stay tuned.

Okay. Now, on with the show. A lot of us just across Australia have experienced quite a lot of extreme weather events and it's no different for the east coast here. We've had some horrific floods which have affected many of us, which makes today's episode really topical.

Now we're talking about weather data and how it contributes to more up-to-date and relevant spatial data and awareness. And while weather is specific to our regions, as we've seen the tips and tricks that we'll be sharing today can be applied by anyone no matter where in the world you're joining us from.

Simon: This episode, I feel like it needs some sort of jingle, so I'm trying to think of what's that crowded house banger?

Wayne: Take the weather with you?

Simon: Yeah.

ALL: Everywhere you go!

Ta: Yeah. That’s the one.

Wayne: I'm sure they're going to cut our singing out there in the editing I’m sure.

Simon: If anyone’s still listening!

Ta: Okay. Now we're all familiar with regular weather data and its use in our daily lives. What's the forecast for today? Is it going to rain? Is it going to be sunshine for my picnic outing this weekend? But, you know, what about other weather data that organisations can leverage for the benefit of their business audience and as the broader community? What about that kind of data?

Simon: Yeah, you're spot on their Ta. For me being from Melbourne, I'm continuously pulling out the Bureau of Met radar app, just to find out, you know, if I'm going to get wet on the way to work or on the way out. And I think a lot of people, the geospatial professionals, often overlook how important that same data would actually be within their own corporate web GIS and actually using that within their organisation.

I think for some organisations having that simple picture of the weather, it can help answer a lot of questions, again like having that radar in your web maps. But I think the point I want to mention here is sometimes it's not just a picture of the weather, sometimes actually getting access to that raw data and a pipeline of that data coming into your system, opens up a whole lot more elaborate workflow, be it Rasta or vector data.

Wayne: Absolutely. And you know, you just said that you were from Melbourne. We should've picked “four seasons in a day” instead of “take the weather with you”, Simon. Now look, there are probably heaps more types of weather data than people really realise. Now there's obviously the type of data that you see the weatherman shows us on the news with the lovely little, you know, isobars and the pointy little graphs and the swirls, humidity and UV indexes. Now, Simon, what other kinds of weather data have you been using? Cause I know you're the weather guy, right?

Simon: Well, not sure about that, but recently I've been pretty obsessed with, the Himawari-8 weather satellites and sort of throwing them into a lot of my maps. They're showing you near real-time views of Australia and all of the cloud fronts across Australia moving around and you can actually dive in and change those pixel values just to filter it down to the ones where it's the serious storm fronts or I believe they're also used it to pluck out bushfires for some scenarios as well.

Wayne: Yeah.

Simon: And different data's going to interest different organisations. So things like frost warnings, like of course that's going to be really useful for agribusiness and for farmers.

Ta: For farmers, for sure.

Simon: Yeah, but then I think a lot of utilities and local governments, they also do a lot of plantations along their waterways and along their parks. And having that frost warning, it might help you plan out, or maybe we'll postpone doing some planting if it's going to be coming up. So, I think it's important across a lot of different industries.

Another one is the fire danger rating. So, that is related to weather and it's something that the Bureau of Met dish out. I think it's going to be important for a lot of organisations that have any people out in the field.

Ta: One thing that I do want to touch on here is data in the moment. So you said that organisations don't realise the various types of data that we have access to. When I think of weather data, I think of historic data.

So I think, for example, you know, just in the situation we're facing here in Queensland, we've recently had some, some terrible, terrible floods, and we had floods 10 years ago. Now that brings us to the point of data in the moment. You know, we focus on the here and now, but what about data currency?

How important is the historical data in either predicting or giving us an awareness of what we need to focus on today?

Wayne: So for planning purposes, this kind of historical data is super, super important. That's just what we can use or what is used to help us construct our, you know, our one in 100-year flood predictions and, that kind of information, allows us to forecast into the future. It allows us to look back against what's happened in the past, and this long-term collection of data has got a whole heap of different applications for it.

So for instance, right now, that kind of flooding data and that rain data is being used by insurance companies so that they can establish, you know, what the flood risk for certain properties are, and then establish their policies appropriately. It's equally as important for climate change so that we can actually measure, gee, it feels hotter this year.

Well, we've actually got the historical data there to actually demonstrate and show that. And we've got decades of data in the store, so to speak from, from large organisations like BOM that can drive this kind of analysis and let us pick up things such as changes in weather patterns, the locations of flood plains and how that evolves over time.

Simon: I think you've, you've really teased out an important aspect there, Wayne. We're very familiar as GIS professionals with temporal data, but weather data is, is super temporal, like it's continuously changing.

Wayne: Absolutely.

Simon: And that change over time, it's not just about the values and what they're doing at a particular point in time. But, how they're trending over time, how does the rainfall compare this season versus last season? What has been the year-on-year change in temperature across either a micro region or a larger kind of state or even the country. And then, understanding how that is actually changing.

Ta: Now this type of data, especially historic data, helps us to look at trends and to predict the future, and then to prepare for it as well. So we're not just talking about rainfall or forecasts, but how will the effects of climate change effect how we work as a community in general?

Think of agribusiness, we mentioned farmers before. So if we're thinking of frost indicators, or even UV indexes, when they're going out to either plant or to harvest, an increase in temperature affects specific crop yields, or coral reefs that's, I mean it's super important, as we're looking at changes in temperature in the water and how it affects our coral reefs. So all of this historic data really helps us to future-proof our day-to-day activities. Can you think of any like additional, real-world examples, I guess that have affected all of us?

Wayne: Well, you know what? I actually have a real-world example here. I've just been working on a major utilities network project, and this is electrical networks. And electricity and water don't really like each other that much right. And so, this is an organisation that works in Queensland and there is, you know, cyclone season. It's a thing, cyclones come every year. We get the same sort of patterns coming across Queensland every year.

And utility networks can't actually do work during those times, where there's a high risk to their staff or a high-risk risk to their assets, they need to plan around this. So our utility networks rely quite heavily on this historic data and on the forecast data so that they know when they are allowed to actually make changes to their network and when they should stop fiddling with it and just let the cyclone fly right on over.

Simon: Should we kind of get into how listeners would be able to get access to those types of datasets?

Ta: Absolutely. So major agencies, within your own organisation, I guess, if you have historic data and you've worked with it prior but, anyone want to name drop anything in particular?

Simon: I think most people will think of the Bureau of Met and quite rightly as the, the authoritative source of weather data here in Australia.

Ta: Yeah. Absolutely. Shout out to BOM for sure.

Wayne: But you don't have to directly go to BOM. I was talking to you Simon just recently, and you told me that there's another way that we can get hold of this lovely weather data from BOM.  

Simon: Yeah, there's a few different ways. So Bureau of Met do provide some really good services, but there's a couple of layers that are beginning to come out in the Living Atlas around cyclones.

Wayne: That's the one.

Simon: Cyclones and those Himawari-8 views as well. Both of those two, you can actually get access to now without having to go through any process, just literally search and add them to it, to your maps.

Ta: Yeah. You can quickly and easily add services from Living Atlas, but what services are we adding when we talk about web GIS? So if I'm going and Living Atlas or adding a service, what are those services?

Simon: The main ones actually feature services. So the Himawari-8 ones, those are actually just image services that you can display in the background and they'll refresh automatically. There's a couple of Esri partners, so WeatherZone and DTN, they provide a range of services that are actually in an Esri feature service format that you can then add to your maps and opens up a few more workflows, such as doing spatial joins or geo-fencing with your operational staff that are out in the field.

Wayne: So a lot of that are open-source format as well, aren't they Simon. So we've got your web feature services and you web map services or open in the OGC formats, the WMS and WFS style formats as well. So it's not just for ArcGIS users. It's for users of all sorts of systems, including open-source systems, as well.

Simon: Yeah, I think it's worth pointing out the Bureau of Met specifically that the GIS to web services, those are the ones that are very easily integrated into the GIS platform. I imagine probably used some of those before.

Ta: One thing that I did want to say quickly is that these types of services can be consumed from very simple web maps through to open-source tools and through ArcGIS Pro.

Wayne: One of the other formats that I've used a lot over the years is a format called NetCDF. It's a bit like a cube of data where you can have multiple measurements that exist over time. It's a multi-dimensional data format.

You can slice and dice and do analysis of various different attributes and weather attributes across time and across spatial regions. So it's a super powerful format, and I've used that in particular with METOC with the Australian Navy.

Simon: That NetCDF format, Wayne like it's quite a scary format for those that aren't familiar with it. You mentioned a cube of data, that already sounds absolutely complex, but I guess Ta, from your experience, in ArcGIS Pro, how easy is it to sort of just get it get started with, with a data set, like a NetCDF?

Ta: From memory and what I've worked with, we'd get the NetCDF data in my ArcGIS Pro, run a bit of analysis, and then I publish that to my portal. So whether that's ArcGIS Online or ArcGIS Enterprise, or I can then access the web feature service or web map servers in my portal, or ArcGIS Online, we share it as a web map if we want. And how do people then visualise that and use that.

Wayne: Well, actually, if we now hark back to that sort of cube nature of the data of it being a data cube, this also applies in ArcGIS Pro there Ta, to our favorite cool little word, voxels!

Ta: VOXELS!

Wayne: Big shout out to voxels as well. So you can format that up in a, in a voxel style rendering and actually see, again, that slice through time and that slice across the spatial regions that you're working with.

Ta: Okay. So I think we've touched on a few things for our tech heads but my biggest concern is what's the end-to-end workflow to get that data out to the public and who can then consume that data in different formats?

Simon: So once you get access to the data, You symbolise it, so if it's something like the radar, you typically would match one of the patterns that people are familiar with from, from the Bureau of Met. You would then publish that and then you would share it within an application to the audience that you need to get to, just like any other layer in, in web GIS.

Wayne: And then of course you'd put that up on a dashboard and host it up in your portal.

Simon: For sure year. Get it out to the masses that need it.

Ta: Yep, so now I understand that for some real-time workflows, some organisations will leverage GeoEvent Server or Velocity to listen into the weather feeds and leverage geo-fencing and incident detection, processes such as alerting when lightening or flood warnings intersect with their assets. So I think that's pretty important, not just from an organisational perspective, but then for anyone who's going out into the field, as well as the community. At least that gives us really good data. That's enriched for good warning systems.

Simon: You mentioned visualisation, do you want this data for visualisation or do you actually want the nitty gritty data for analysis like that NetCDF's. Wayne do you wanna just kind of walk us through what you kind of feel for it around that?  

Wayne: It's horses for courses it's for what you need it for. So if you're doing analysis and you're doing historical analysis, or you're doing predictions into the future, then that NetCDF format, that cube style format and the fact that you can scroll back and forwards in time and space through it, is super, super important. It can, add insights that you just can't see with instantaneous data.

But then there's instant and live data, just like live traffic data can change how you route and move around your world. Live weather data can have the same effect. So, where's my local park and what's the weather going to be like, am I going to get wet when I go and take my dog out for a walk? And that can be just as useful right out in full-blown enterprise systems.

Simon: Another theme that you touched on Ta was that potential within organisations, it's all good and well to kind of get excited about the data but, how can you actually relate this data back to the people in your organisation and what they're doing?

Ta: We have to consider where the data would be useful for our internal workflows. So how do we bring that data into our systems? We've talked about, OGC formats web feature services, web map services.

And then we have to think about the results that we want to achieve. Is it a snapshot of information or are we looking at analysing and analysis? If we're integrating that data with our workforce systems, you know, people are going out into the field to either look at crops, to do crop analysis, perhaps soil analysis. We have to be able to update them so that they can choose the best days and times to go out and do their work.

Finally, where is that information going and who is it benefiting? And as Simon said, you can reach out to him via Twitter or LinkedIn, if you want to know a little bit more about how to use weather data.

Wayne: We'll be publishing his, uh, his home address on the podcast website as well won't we.

Ta: Oh man. Absolutely.

Simon: I reckon if someone did turn up and knocked on the door and wanted to talk about GIS and weather, I'd probably entertain it.

Wayne: Oh you've gone and opened the doorway now!

Ta: Oh, my goodness.

Wayne: All right folks, that's been another lightning quick episode. See what I did there. And it's been full of tips and tricks to help get you started with your weather data in your GIS.

Ta: Well, to help get you started, come rain, hail or sunshine we'll include a bunch of useful links on the website at GISDirectionsPodcast.com.au.

And we'd also love to hear from you guys, so don't just shout out Simon alone, but please come into the website or connect with us through either Twitter or LinkedIn or any of our socials and make sure you follow us wherever you get your podcasts and keep those five-star ratings coming. We'd love them so much.

Wayne: We do indeed. Thanks for joining us.

Simon: Until next time.

Ta: Stay safe and happy mapping.

Disclaimer: The views and opinions expressed in this podcast are solely those of the hosts and do not necessarily represent the views or opinions of Esri Australia.