Unpacking ArcGIS Enterprise 11.1 – extended cut

Maggie: Being able to visualise your data in new and easier to understand ways I think really allows you to gain additional insights and then, take action on those insights. 

Disclaimer: This podcast is brought to you by the team at Boustead Geospatial productions, in conjunction with our affiliates - Esri Australia, Esri Singapore, Esri Malaysia, and Esri Indonesia. To get your hands on more short, sharp and immediately usable resources, head to the GIS Directions Podcast website, and check out the show notes. 

Wayne: Welcome to GIS Directions. I'm Wayne Lee-Archer.  

Simon: And I'm Simon Jackson.  

Wayne: And joining us today we’re very excited to have Maggie Busek, Product Manager with Esri Inc’s ArcGIS Enterprise team. Welcome along Maggie. 

Maggie: Thanks, it’s so great to be here.  

Simon: So Maggie, there's this new long-term support release of ArcGIS Enterprise that's now available, that's 11.1. So today we wanna kind of focus on that core ArcGIS Enterprise stack, talk about all of the new cool stuff that's available in that release.  

We know there's all the other Enterprise roles like GeoEvent and Knowledge Server, and you know, also the Kubernetes. But today, let, let's kind of jump into what's new in ArcGIS Enterprise 11.1. 

Maggie: Sounds great.  

Simon: Maggie, before we talk about the what's new, can you tell us a little bit more about your role and kind of what you get up to in the ArcGIS Enterprise team and which bits you focus on?  

Maggie: As Wayne said, I'm a product manager on the ArcGIS Enterprise team. 

And as our listeners probably know, it's a big product. So, I don't do everything for Enterprise. I have a few areas of focus. Things that I think a lot about are things like upgrades, the Enterprise Portal, accessibility, I also work on our Enterprise beta program, and looking at new things all the time to take onto my plate. 

Simon: That beta program interests me. So that, that's something that came along at 11.0, isn't it? Where there was suddenly there's option for the administrator to decide if they wanted to enable new features. Can you tell us a little bit more about how that works?  

Maggie: So, at 11.0 we introduced two new concepts that related to beta and beta features, beta programs. So at 11.0, we started the new version of our Enterprise beta program. And through the Enterprise beta program we put out a couple of releases in advance of the final version of the software for early testing. 

This is a great opportunity for users to get a look at new features that are coming, prepare for those, maybe do some training internally to get ready for those new things that are coming in the next release. And that started at 11.0 and it’s something that we've continued with 11.1. We're even looking at our beta program for 11.2 currently, and then also at 11.0 we introduced this concept of beta features as well. 

So these are features that are in beta themselves, but are in a final release of the software. So at 11.0 we had things like service webhooks that were in beta. We had windows containers for notebook server which were in beta. And these have a different promise for support around them than all of the other features in the final release. 

But it's a great way to test things that are being finalised and going to be fully fleshed out in a following release. So at 11.0 we have those beta features. We don't have them in 11.1 because we only have beta features in short-term support releases, and 11.1 is a long-term support release. 

But we look forward to potentially having other beta features in a future short-term support release.  

Simon: And you mentioned the long-term support release and short-term support release. Can you just remind me, we have 11.1, which is a long-term support release. What does that mean? Is that's a longer support period, is it? 

Maggie: Yes, that's right. So any given version of Enterprise has a support window associated with it. We determine this with our product lifecycle and so we have this concept of short-term support releases and long-term support releases. Short-term support releases, have technical support for three years, and we provide patching for a year and a half, and then long-term support releases, you might guess, have a longer window of support, and those have six years of technical support and four years of patching associated with them. 

So 11.1 is our most recent long-term support release. And typically, we see users moving to both our short-term and long-term support releases, but often we see even wider adoption of our long-term support releases just because of that longer support window. 

Simon: The new stuff that's there at 11.1, there's quite a big list, so I feel like we should limit the scope to Windows and Linux deployments, cause there's a whole bunch of new stuff in the Kubernetes 11.1 release as well. And again, I guess ArcGIS Enterprise, as you said, Maggie, it's a huge product and I feel like we should narrow in on the traditional base ArcGIS Enterprise deployment and the, the portal and the data store the GIS server. 

What are some of the, the new things within that base stack that you're particularly excited about at the 11.1 release?  

Maggie: I think zooming out a little bit, thinking about our high-level message and focus for the release, with each release of Enterprise, we try and have a key message and motivation behind it. 

And for 11.1, our key focus is the ability to enable our users to make high quality decisions quickly. So this is something we want Enterprise to support regardless of what version you're on. We know Enterprise is this foundational product that's at the centre of all of your GIS workflows, and it enables you to gain insights and inform really important decision making. 

And at 11.1 we really focused on this message, and we have a few features that I think speak really well to that.  So, new enhancements in our map viewer and scene viewer, building those out to add additional context to your data through new visualisation tools and adding some kind of real-world context to your data. Some means to do your work more efficiently in the Enterprise portal as well.  

And then we also have some pretty exciting extensibility features that came out in 11.1. We know Enterprise out of the box can do a lot of incredible things and it works very well for many of our users just out of the box, but there are users that want to be able to customise and extend the functionality of Enterprise with extensibility tools. 

And so, we built out this story with webhooks and custom data feeds at 11.1. and so those are the things I'm probably most excited about.  

Wayne: One of the things that interests me is that custom data feeds that you just mentioned, Maggie, exactly how's it gonna work, how's it gonna be useful? 

Maggie: So custom data feeds we are really excited about because we see this as the opportunity to kind of unlock data that might have been hard to pull into Enterprise previously. And so how this works is you can work with any data source, even ones that aren't natively supported by ArcGIS. It could be a MongoDB database, it could be an elastic search database, it could be a GeoJSON feed on your city's website. 

And in order to access them and pull them into your Enterprise system, what you'll do is configure a custom data provider and that custom data provider will enable you to have the ArcGIS server component of ArcGIS Enterprise go out and fetch from that data source and pull it in to Enterprise as a read only feature layer. 

From there it takes on the security of, you know a normal feature layer within the ArcGIS Enterprise, and then you can interact with it within Enterprise, pull it into maps and things like that. So we're really excited about it because it's number one data sources that aren't necessarily natively supported with ArcGIS traditionally and allowing you to pull in that data from new sources without having to process it. Maybe do some ETL work prior to getting it into Enterprise.  

And then you also don't need to be the owner of the data that you are working with and pulling into Enterprise with these custom data feeds. So I mentioned that, GeoJSON feed on a city's website, you can configure a custom data provider for that GeoJSON feed and pull it directly into Enterprise without actually being the owner of that GeoJSON feed. 

So, one consideration I do wanna point out is you will need a developer in order to help you configure that custom data provider that's required to pull in that data source. So I know many listeners probably have developers within their organisation. 

Maybe you don't and you need to work with someone else. So just figuring out what the right resource for that would be.  

Simon: I guess the other one that you mentioned, Maggie, was webhooks. And from what I've read, it kind of is broken down into these organisation webhooks, but then also feature and geo processing webhooks. Can you elaborate a little bit more on those and how they might be useful for a GIS administrator? 

Maggie: So webhooks are not necessarily a new concept to the ArcGIS system and organisational webhooks we actually introduced a couple of releases ago. But at a very high level, essentially what a webhook does is it will have a particular action happen within Enterprise for an organisational webhook, for instance, maybe an item’s being created, maybe a user's being deleted, things like that, and it'll send a notification to an administrator. 

And so it's a proactive approach. It's a, ‘don't call us, we'll call you’ approach that, again, is sending those proactive notifications when an action happens. So organisational webhooks we've had, we have other types of webhooks throughout the ArcGIS system, but what we're really excited about in 11.1 is the introduction of, service webhooks. 

So we have both feature service webhooks and then geo processing webhooks. With feature service webhooks, these are going to send notifications when, what we call CRUD events happen. So Create, Read, Update, and Delete. If you have a feature that's been deleted or a new one's that's been created, you'll get a notification if you have that webhook configured. 

And then geo processing webhooks will send notification when a geo processing job is finished. So we all know geo processing jobs can finish in a variety of ways. Maybe they complete successfully, that's great, but they could also fail. They could be ended. And so you'll get a notification regardless of what that kind of finished state is. 

And so that proactive notification we're really excited about. You're not gonna have to spend so much time going back and clicking, is this done yet? Is this done yet? What's changed? You'll just get those notifications and kind of be able to focus on the important work that you're doing instead of going back and checking on things. So we're really excited to expand that story of webhooks.  

Wayne: That is super exciting. It sounds like a really handy integration point to integrate with third party utilities. We do a lot of integration with, you know, asset management systems, for example, in the utility space where I work. 

This sounds like the perfect sort of segue into how we can actually integrate with some of these solutions. I'm really excited about these webhooks.  

Simon: You mentioned integration there, Wayne, and I know you do a lot of work in that space and I feel like a combination of the custom data feeds into sort of third party systems, pulling data out of those systems and surfacing it and ArcGIS just a, you know, a regular feature service. 

But then also the opposite, right? Being able to listen out for when someone creates a feature in the GIS and send a payload to another system to also generate or get an asset ID and things like that. It feels like it's gonna open up a lot of options.  

Wayne: Those are my key features. I'm taking those ones and I'm running with them. Those are my two key features from this release.  

Simon: I guess Maggie, I’m, I'm more on the front end and, and quite interested in some of those improvements to the map viewer and the scene viewer, like in particular, I saw that chart style appear in Enterprise 11.1, which, working with census data over here is a great style to sort of, get an understanding and breaking up that census data. 

Have you sort of seen any other improvements in the map viewer or scene viewer that are worth shouting out about?  

Maggie: We're really excited about that chart styling as well. We're also excited about some of the aggregation styling enhancements that we've had come to the map viewer in 11.1. So things like binning has been introduced in 11.1. 

We've also seen improvements for heat maps, so we're getting better performance with heat maps now, we support popups with heat maps, so kind of expanding the functionality there and thinking back to that key message of, making high quality decisions quickly. 

Being able to visualise your data in new and easier to understand ways, I think really allows you to gain additional insights and then, take action on those insights. In the scene viewer, as you mention, we've added some exciting things to give more real-world context to the scenes you're interacting with, so, things like weather effects are, I think really cool.  

Dimensions – being able to see those kind of real-world measurements for the different things you're interacting with within your scene are super exciting. So yes, lots of new enhancements within the map viewer and scene viewer to look forward to. 

Simon: Enterprise obviously has an in, integrated with the ArcGIS Living Atlas and there's the open street map building and tree layouts. Not just 2D, but actually kind of 3D representations of buildings and trees. And the open street map data for Australia is really good. 

So being able to kind of, just compliment your existing scenes with that layer is really worth looking at.  

Wayne: The secret, and not so hidden developer in me spotted another feature in there that may be useful for some of our people out there, and this is the Arcade Editor. Tell me a little bit about the Arcade Editor. This has been a bit of a game changer for me.  

Maggie: Yep, so we have made some substantial enhancements there at 11.1 as well. We've been focused on how to make it kind of easier and more efficient to write Arcade with Arcade Editor, and so now with the IntelliSense functionality, we've added that and we're excited. It provides suggestions as you're writing your arcade, prompting you, and making that easier with auto complete functions, helping you find errors and things like that. 

And then just from a user experience perspective, we've also modernised the layout for the Arcade Editor. So it's a more easy look and feel when you're interacting with it.  

Simon: One of the other ones that we know a lot of our clients have been asking about is, so some of 'em have things like Google BigQuery and Snowflake, and I, I saw that 11 has support for tapping into those data stores. 

Maggie: Yep, so cloud data warehouses are something that we've supported since 10.9.1. We've heard the interest in being able to leverage those data sources and pull that data into ArcGIS Enterprise. And so starting at 10.9.1, we introduced support for cloud data warehouses from BigQuery like you mentioned snowflake and then also Redshift. And, we've been expanding this story over the last couple of releases. Originally, you were only able to publish map image layers from those cloud data warehouses. 

But we've expanded this story in both 11.0 and 11.1, and now you can publish feature layers from BigQuery and now Snowflake at 11.1.  

Simon: Let's kinda shift gears. I know there's plenty, plenty more stuff in 11.1 that we could talk about, I wanna sort of talk about how we can get to 11.1 and, from what I understand Maggie, there's two ways of looking at it, so if you're on an older version and you have a, a bunch of services that you've published over that version and maybe even previous versions and then you've also got a, a range of content that's sitting in your portal, apps and web maps and dashboards and so forth.  

When you're moving to 11.1, what are some of the key questions you should sort of be asking yourself before rushing ahead and clicking the upgrade button?  

Maggie: So upgrading is something we always encourage you to think through before you do it. We wanna be taking backups and just have a plan, making sure that it's a thoughtful process. 

And this is still true if you're upgrading to 11.1. there might even be a little bit more pre-planning depending on which version you're coming from. I think I wanna zoom back to the 11.0 release a little bit because we are building on some of those significant changes that we introduced at 11.0 in this new version of 11.1.  

So at 11.0, we removed support for the ArcMap runtime, this is what enables you to publish services from ArcMap to ArcGIS Enterprise. And so we know many of our users have moved away from ArcMap. They’re on ArcGIS Pro, which we're really excited about. But if you’re still using ArcMap and you're still using those services within ArcGIS Enterprise, you're going to have to do some work to migrate those services to ArcGIS Pro prior to moving to 11 .0, or onwards like 11.1. 

Wayne: An excellent reason to get onto Pro.  

Maggie: Yes, exactly. We’re always encouraging that.  

We also know this removal of the ArcMap runtime is a pretty significant one, and it was a big breaking change at 11.0. And so we tried to couple that with some other significant changes just because we knew there was gonna be some planning associated with moving to the 11 X generation anyway, and wanting to have some significant transitions happen all at once instead of having big changes with every coming release. And so some of our older app and EMP templates have been retired at 11.0 and onwards into 11.1. So things like classic StoryMaps templates, some of the older configurable apps templates. 

ArcGIS dashboards classics and so we have a very thorough list of all of the considerations that you need to think through prior to upgrading to 11.0 or 1.1, from those older versions. But we also tried to make this process hopefully as easy as possible with some in-app tooling. And one really big piece of it is those ArcMap-based services.  

If you're still using them, some ArcMap-based services will migrate automatically when you upgrade to 11.0 or 11.1, your feature layers, your map layers or feature services, your map services. But some things you have to actually manually migrate. So things like geo processing services aren't going to automatically migrate. We have some tooling to help you migrate those automatically, like I said, and then also take an inventory of which services you're going to have to manually migrate. 

We introduced this tooling back at 10.9, built on it in 10.9.1 and then continue to offer it in 11.0 and onwards just to help aid in that process. For the apps and app templates that we retire, the level of migration really kind of depends on what it is, for the Esri StoryMaps classics templates, you are gonna have to rebuild those StoryMaps. 

So that's a little bit more migration, but things like the classic dashboards, when you open them within 11.1 or 11.0, that will have automatically migrated. You don't have to do anything to migrate content to new the new ArcGIS dashboard. So, those are things to think about. We don't want this process to be hard or scary, but just something that you have to be thoughtful about and kind of map out in advance. 

So one recommendation we have is going to 10.9.1 prior to going to 11.0, or 11.1. We know many users are already on 10.9.1. but 10.9.1  is a really good release because it has these older versions of some of the things we retire at 11.0 and onwards, it supports both ArcMap and ArcGIS Pro based services. 

So it's a great place to do that migration work in advance of going to 11 X, so if you're on a release prior to 10.9.1, we recommend going there prior to going to 11.0 or 11.1.  

Wayne: That is a super tip. So that's sort of like the proving ground where you can actually do some of that migration work that's gonna need to be done. I think that's a great tip. Thank you so much for that one, Maggie.  

Maggie: Absolutely. And we, again know this could be a big transition and so we have a lot of documentation and resources around this work and recommendations of moving 10.9.1 first. 

So we have some blog posts and product documentation that definitely are a good thing to take advantage of as you think about this process.  

Wayne: Great. Now, we, we hear, that seeing as some of the older apps have been deprecated. Is Web AppBuilder still hanging on in there in 11.1?  

Maggie: Yes, we have Web AppBuilder and Experience Builder as well, it's kind of newer version in Enterprise 11.1, so that you don't need to worry about migrating quite yet, but you should be thinking about moving those workflows that you have been doing with Web AppBuilder onto Experience Builder where it makes sense.  

Simon: Maggie, I feel like under the hood with the upgrade process, your team's done something, some improvements to kind of speed things up. Like I've noticed going from 10.8 to 10.9 to 11.0 to 11.1 on my lab environments that have quite a lot of content, I've noticed for one, kind of the GUI seems to have more options, but also, watching the status bar seems to have got quicker over the versions, is that, what's happening under there?  

Maggie: Yes. that's been something that we've put a lot of effort into over the past few releases. We want you as users to be able to take advantage of all the new and exciting stuff with each release. 

And so we wanna make that upgrade process as easy and quick as possible. And so we've been thinking about how to do that and so we introduced a lot of work at. 10.9.1 to improve that underlying framework that powers our upgrades and make that process faster, but we're really seeing the full kind of realisation of this work at 11.1. 

So, we also have been aware of the fact that, sometimes you wanna see where you're at in the process when you're upgrading.  

When you're upgrading Enterprise, you go component by component. And sometimes it might feel like a little bit of a black box of like, okay, I'm upgrading portal for ArcGIS, where in this process am I at? So a couple of releases ago we introduced a means to check where you're at in the process. 

You know, if there's a few steps, it'll show you incrementally what place you're at in this upgrade, so you have a sense of how you're progressing, we think that improves the experience and we're thinking about how we might move that to other components in the future. 

Like I said, it's specific to portal for ArcGIS for now. But again, thinking about what this might look like in the future across Enterprise.  

Simon: That's great work. So something me and Wayne have debated over the years is what's best practice for upgrades, Like, like doing it in-place upgrade to an existing environment and potentially having some downtime for users or having to sort of do it over, out of office hours versus that approach of deploying a fresh environment from scratch and migrating content across and then shifting people to the new environment, is there a best practice around that? Or is it kind of case by case?  

Maggie: It really depends on the organisation, there's different considerations that different Enterprise users really need to make when they're upgrading. But it really depends on kind of your organisation's needs, what type of downtime you're working with. We know different Enterprise users have different service level agreements with their users. 

Yeah, it's something for you to consider on an organisation-by-organisation basis.  

Simon: One last thing I was gonna ask you about Maggie is have you seen much around people sort of moving away from clicking through doing upgrades from GUIs to moving to kind of DevOps workflows? How much of a kind of shift have you seen from the global users that are upgrading? 

Maggie: Again, that's another thing that depends on the organisation and the makeup of your team.  We are actively improving all of those different options for upgrading, just to make sure regardless of what the right fit for your organisation is, you have a good process for upgrading. 

And so we see it pretty spread out across our different users. Again, just kind of dependent on what works best for any given organisation. 

Simon: You mentioned backups, Maggie, so, what's your recommended approach for before you even start an upgrade? How should someone back up?  

Maggie: Wayne, I'm actually kind of interested to hear your perspective on this, cuz from my understanding, you do a lot of upgrades within, the Australia upgrades team. So I'm interested in what your experience out in the field has been.  

Wayne: Wow, I didn't expect to be on the receiving end of the questions. Excellent! This is a bit of a rebellion going on. You know, we've got a lot of tools in the toolkit in this space. Migration between environments, ah, isn't just about when we're doing upgrades, migration between environments is also about when you might be moving between say, a dev and a test or a prod and a pre-prod environment.  

We still need to migrate content between places. This is not just about backups for backup purposes, but it's also using backups for migrating content between environments as well. 

And, we've still got the old staple that we keep turning to, and that's WebGISDR. The WebGISDR toolkit really has stayed with us throughout the years. So, I think my tip as far as, as that is concerned is to really lean on WebGISDR.  

If we can get the routine in place that makes WebGISDR work, get those snapshots and those backups happening regularly, get them stored on different storage, so don't save them on your C drive so that when you fail, you're happy. All of those kind of things taken into consideration. 

WebGISDR is really gonna be your savior, not just when something goes wrong and everything falls in a, a smoking heap. But also when you're moving between environments, you can actually use WebGISDR very cleverly then.  

Maggie: Yeah, that’s great advice.  

Wayne: Alright, we've talked about upgrades, what, Maggie, can you tell me about the future?  What should we expect to see in 11.2, 11.3? what’s coming next Maggie? 

Maggie: Well, I want to take this opportunity to plug the beta program for Enterprise, this is a great way for users to get to look at things that are coming. As I mentioned, we're getting our 11.2 beta program together and we have some exciting stuff that users will get to test by participating in that program. So I'm gonna point to that. 

We also often talk about this in our road ahead sessions at our different Esri conferences so that's a great place to hear what's new.   

Wayne: Thank you so much for that coy answer there, Maggie, but I understand. Let's give 11.1 it's day in the sunshine is what I'm hearing.  

Maggie: Exactly.  

Simon: So just to wrap up, Maggie, what are your top three highlights for 11.1? 

Maggie: I think, custom data feeds we're super excited about, I know that was something Wayne was smiling about. So I'm really excited for that to be coming. Service webhooks are another big one that we see a lot of possibilities with, both for feature services and then geo processing as well.  

And then I think some of the enhancements in the scene viewer, I just find really cool things like the weather effects and stuff like that give that real-world feel to the scenes you're interacting with there. So some of that I think is very flashy and exciting. 

So yeah, super excited about 11.1 and there's a ton that we of course couldn't cover in our podcast today. But I would say those are my top three.  

Wayne: I'd like to take this opportunity to thank you for joining us today, Maggie.  

Maggie: Thank you so much for having me.  

Wayne: And so that's it, folks. Some short, sharp, and immediately usable features to update your organisation's capability.  

And to get you started with these tips and trick, we've added all of the resources we've spoken about today to the website, that's gisdirectionspodcast.com au, and that includes all of the information that you'll need to get started with ArcGIS Enterprise 11.1.  

Simon: And look, we'd also love to hear from any tips from our listeners, you the ArcGIS community, jump onto the website and send them through or connect with us on LinkedIn and hopefully we can feature some of those tips as part of upcoming episodes. 

Wayne: Wonderful stuff. Stay spatial.  

Simon: Until next time. 

Disclaimer: The views and opinions expressed in this podcast are solely those of the hosts and guests and do not necessarily represent the views or opinions of the Boustead Geospatial Group of companies.