Free and open source software (FOSS) is increasingly being used by public sector organizations. Recently, the Canadian government issued the directive that wherever possible open source software and open standards should be leveraged over commercial off-the-shelf (COTS) software and proprietary technology. Specifically, their Directive on Management of Information Technology, which took effect on April 1, 2020, stipulates that “where possible, use open standards and open source software first”[1] and “all source code must be released under an appropriate open source software license”[2].
At the provincial level, British Columbia has long been a champion of open data and open source software. BC’s Citizens’ Services’ Guidelines on the Use of Open Source Software provides guidance to users who wish to adopt open source technologies. At the municipal level, PSD Citywide’s annual Geospatial Maturity Index found that in 2019 just over 20 percent of respondents use open source GIS software within their organization, a number that likely will grow as open source GIS gains popularity.
As governments become progressively open and budgets are increasingly constrained, the time is ripe to start considering open source software as a viable alternative to COTS solutions. This guide is meant to give public sector organizations some background information about open source GIS software so they can make more informed decisions moving forward.
Defining Open Source Software
Fundamentally, open source software can be obtained for free whereas COTS software is mostly sold with strict licensing. One can download and install open source software on innumerable workstations, whereas COTS software often limits the number of workstations per license. With open source software, one can also download the source code and modify or customize the application since it too is freely released and distributed. With COTS software, the source code is usually a company secret, rarely if ever seen by end users.
There are also differences in the development process of open source software versus COTS software. Open source software is developed by dozens or even hundreds of end users, academics, “paid volunteers,” and companies, whereas COTS software is developed by a finite group of employed developers. The main difference here is that with more eyes scrutinizing open source code, bugs and security issues are often (though not always) identified and addressed quicker within open source software.
Additionally, with the larger workforce supporting the applications, mature open source software often has a rolling release, meaning that the software is updated and released more regularly, possibly daily. In contrast, except for the bug fixes, major releases of proprietary software can be years apart.
Another interesting caveat about open source software is that it cannot contain proprietary code under a different license. According to the criteria for open source software established by the Open Source Initiative, an organization that promotes the use of open source software, the entire application must be under the same open source license, such as the MIT or GNU General Public Licenses. This implies that all features within the software must be freely available to use and modify, so end users will never be prompted to pay for access to specific tools or features – unless, of course, the software is licensed with a free and paid tier.
Conversely – and this is where it gets nebulous – COTS software can actually contain open source code if the license for that code permits redistribution, modification, and sale, which is permitted under many of the open source licenses. For example, the MIT License states that:
“Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so…” [3]
The implication here is that many of the software developers building COTS software will leverage open source tools under this licensing, be it a database management system, a specific library to visualize graphs in a web application, or a wiki application to facilitate documentation. Consequently, we are all users, knowingly or unknowingly, of open source software. The majority of web servers online are Linux-based, the operating system in our smartphones contains open source code, and even some of the very expensive software we purchase likely contains some open source code.
Benefits of Open Source Software
Open source software has a number of benefits for public sector organizations. Of course, the lack of licensing fees is a primary benefit. It is not uncommon for public sector organizations to pay upwards of $100,000 annually for GIS software licensing, so the prospect of paying $0.00 annually is often quite appealing. Although the software license is free, one should be aware that there are always still costs associated with training, implementation, and development time to customize the solution. It should also be noted, however, that these costs may exist even with COTS software.
Beyond the main financial benefit being a lack of license fees, there are a number of secondary benefits, namely eliminating the time, effort, and red tape associated with software purchasing and the ability to put software on every desk without additional costs. Over time, migrating to open source software also eliminates any surprise license fee increases. As a result, open source software tends to be more sustainable over time by minimizing operating costs and removing barriers to software adoption.
In addition to cost, open source software has a number of additional benefits. Functionality and technical capabilities are a key benefit. Since the software is developed by and for end users, it can be more intuitive and feature rich. It is often more customizable and supported through a large user group that includes end users and core developers. Since mature open source software is often built using the most up-to-date standards, the software tends to have a high level of interoperability, which means that except for highly proprietary file formats, most open source software can read and write any file formats from COTS software.
Lastly, another major benefit is extensibility. Through customization, the software can be extended with new features and capabilities.
Open Source GIS Software
The open source GIS software landscape is vast. For example, whereas there are only a few mature open source office software suites (e.g. OpenOffice and LibreOffice), there are dozens of mature GIS applications, many of which have been in active development for decades and are the core modules within even COTS GIS software. Open source GIS software can be categorized into seven broad categories, namely (1) content management systems, (2) metadata catalogs, (3) desktop applications (and associated geospatial libraries), (4) web-based GIS, (5) GIS servers, (6) spatial databases, and (7) mobile data collection applications.
(1) Content Management Systems and (2) Metadata Catalogs
Geospatial content management systems (GeoCMS) and metadata catalogs are closely related technologies that are increasingly fused into a single solution. A GeoCMS will display an organization’s GIS layers, documents, and other multimedia on a map and perhaps allow users to create customized map layouts.
A metadata catalog is a database application containing information about each layer (e.g. date created, owner, data license). These catalogs often conform to international or national standards, such as ISO 19115/-1 for vector data, ISO 19115-2 for raster data, or the Content Standard for Digital Geospatial Metadata (CSDGM).
For example, GeoNode and GeoNetwork are two applications that function as both a metadata repository and a portal for geospatial data. The fundamental purpose of these applications is to make data discoverable, and in so doing allow users to interact with the data on a webmap, through web services, and/or via downloads. These are core applications within any enterprise GIS.
(3) Desktop GIS Applications
There are numerous open source desktop GIS applications on the market, and many of which have been in development for decades. These mature applications are often quite robust and include a collection of geoprocessing tools that rival COTS GIS applications. The oldest and most widely used among them is GRASS (Geographic Resources Analysis Support System), which has been in development since 1982. It currently has over 350 core modules that can perform even the most advanced GIS functions, such as complex 3D point cloud analysis and terrain analysis. However, the learning curve to work with GRASS is definitely a barrier for some as it does not have the same look and feel of more common GIS desktop applications, thus it is often used most by advanced analysts and researchers.
A similar desktop GIS application is SAGA (System for Automated Geoscientific Analyses), which has been in development at the University of Hamburg since 2001. Like GRASS, SAGA has a modular framework with hundreds of tools to conduct a wide range of analysis, including terrain analysis, simulation, image classification, geostatistics, and hydrological analysis. It is used most commonly as a remote sensing tool since the majority of its modules focus on image processing and analysis.
Again, like with GRASS, SAGA does require a learning curve for traditional GIS users since it is more akin to remote sensing software in its layout and functionality. Novice users may also be overwhelmed by the sheer number of processing modules available within the application.
Perhaps the most commonly used open source desktop GIS today is QGIS, which has become a major competitor to COTS GIS software since its development began in 2002. Today, QGIS is in version three and it is a robust application with hundreds of core geoprocessing tools and functions. It also integrates tools from GRASS, SAGA, and other 3rd party open source and proprietary solutions to extend its functionality.
Anyone can further enhance its functionality by building custom plugins and share them with the whole user community via the Plugin Library. Since QGIS’s layout and functionality is similar to that of traditional COTS GIS software, it is often easy for users to adapt and migrate workflows. This process is aided by high quality documentation, various third party training options (e.g. workshops, books, online tutorials), and a supportive user and developer group. Hence, the migration to QGIS can be quite painless with a wide range of tools and extensive support available.
(4) Web-based GIS
Web-based GIS (or webGIS) is a prominent feature of most enterprise GIS deployments today. Almost every public sector organization wants some form of web-based GIS application to enhance public engagement and to share open geospatial data. There are two open source frameworks that can be used to build advanced web-based GIS applications, namely OpenLayers and Leaflet. Both are built using JavaScript, a programming language used mainly for web application development, and each give developers access to a range of basic functionality to build dynamic webGIS applications.
For the standard end user without programming skills, QGIS offers a plugin called qgis2web that converts any map project into a web application. Other applications leveraging these webGIS frameworks include GeoMoose and Map Bender, two applications that allow users to deploy webGIS applications without writing code. Thus, webGIS technologies are becoming increasingly accessible in open source GIS and there are a range of options available for enterprise deployments.
(5) GIS Servers
GIS servers support webGIS applications, such as mapping applications, GeoCMS, and geospatial metadata applications. The fundamental purpose of a GIS server is to share geospatial data using a series of standardized services, which were established by the Open Geospatial Consortium (OGC).
For example, when serving aerial photographs, one would typically use a Web Map Service (WMS) that can serve imagery quickly as a series of smaller tiled images. For vector data (e.g. point, line, polygon), the data can also be served as a Web Feature Service (WFS), which has various capabilities, notably to facilitate editing. There are various other standard services available, and most GIS servers, open source or COTS, implement them all.
There are three main open source GIS servers on the market, namely Geoserver, MapServer, and QGIS Server. These applications support all standardized web services, advanced authentication, and other industry standard functionality. Although setting up a server usually takes advanced IT knowledge, the documentation for these applications, notably for Geoserver, is superb. As with any GIS server, there is always a learning curve that often requires professional support and training.
(6) Spatial databases
There are two core architecture types for geodatabases, namely client-server and file-based. Client-server architecture is similar to COTS platforms like Oracle Spatial and Microsoft SQL Server, and are an integral part of an enterprise GIS. There is usually a centralized relational database management system (RDBMS) that manages data access roles/users, backups, and other tasks. Users can access the data via a number of clients, such as a desktop, mobile, or webGIS platforms.
The premier open source spatial RDBMS is PostgreSQL with the PostGIS spatial extension. PostgreSQL, or Postgres for short, was first developed in 1980 at the University of California, Berkeley, and the initial release was in 1996. Although the RDBMS is used widely today, it was adopted by the GIS community in 2001 with the first release of PostGIS, a spatial extension.
With PostGIS, users have access to hundreds of geospatial functions. In addition to centralizing data within an organization and managing data access, PostgreSQL/PostGIS has the added advantage of streamlining spatial analysis. Although an essential component of an enterprise GIS, PostgreSQL/PostGIS requires considerable expertise to configure properly, but once set up even novice users can easily access an organization’s data through a desktop client.
Open source file-based geodatabases are quickly becoming popular alternatives to the proprietary Esri shapefile format. Geopackage (.gpkg), for example, is the most popular file-based geodatabase. It can store multiple layers and customized styles, so anyone loading the layers will see the exact same map. Geopackages can also be shared easily since they are a single file. The other advantage for smaller organizations is that users get most features of an RDBMS without the requirement of a server. Either way, the use of a database system is critical within an enterprise GIS.
(7) Mobile data collection
Recently, GIS has moved to the field thanks to a range of data collection applications. On the COTS side, Esri has two main applications, namely Survey123 and Collector for ArcGIS, while other applications like Fulcrum App have gained popularity, especially among those not committed to the Esri platform. These COTS applications allow users to create customized data collection forms and collect data either online or offline.
As a newer development in the GIS sector, many of the open source mobile applications are fairly new. QField, for example, allows users to create a project in QGIS and then transfer it to an Android mobile device. Data can be collected online or offline but the results have to be manually transferred to a desktop, which can be cumbersome for some users.
Alternatively, Lutra Consulting developed a subscription-based application called Mergin that can be used to store QGIS projects in the cloud. These can be accessed from the desktop or from a mobile device using their Input App. For many users the costs associated with something like Fulcrum App or Mergin is acceptable, especially when licensing and service costs for an enterprise GIS are minimal.
Enterprise Open Source GIS Software
Having reviewed the various types of open source GIS applications, it is possible to assemble an enterprise GIS solution. For example, PSD Citywide’s Enterprise GIS merges Geoserver, PostgreSQL/PostGIS, QGIS, QField, and customized software while offering extensive support and training. It is this type of solution that allows public sector organizations to build a more sustainable GIS department without the burden of excessive software licensing costs. For many organizations, taking the plunge into open source GIS is a big step, but a necessary one to advance their organization’s GIS and asset management strategy.
[1] https://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15249, C.2.3.8.1
[2] https://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15249, C.2.3.8.4
[3] https://opensource.org/licenses/MIT
