Mapshaper to Convert, Edit, and Join Data
Like GeoJson.io, Mapshaper is a free, open-source editor that can convert geospatial files, edit attribute data, filter and dissolve features, simplify boundaries to make files smaller, and many more. Unlike GeoJson.io, Mapshaper doesn’t have drawing tools, so you won’t be able to create geospatial files from scratch.
This free and easy-to-learn Mapshaper web tool has replaced many of our map preparation tasks that previously required expensive and hard-to-learn ArcGIS software, or its free but still-challenging-to-learn cousin, QGIS. Even advanced GIS users may discover Mapshaper to be a quick alternative for some common but time-consuming tasks.
Import, convert, and export map boundary files
You can use Mapshaper to convert between geospatial file formats. Unlike GeoJson.io, Mapshaper also supports Esri Shapefiles (which is a folder of individual files with the same name, but different extensions), so you can easily convert a Shapefile into a web-friendly GeoJSON.
- Navigate to Mapshaper.org. The start page is two large drag-and-drop zones which you can use to import your file. The smaller area at the bottom, Quick import, uses default import settings and is a good way to begin.
- Drag and drop your geospatial file to the Quick import area,
or use our sample Shapefile of US state boundaries.
This is a
.ziparchive which contains a folder with all necessary files.
Note: If you want to import a folder, you need to either select all files inside that folder
and drop them all together to the import area, or create a
- Each imported file becomes a layer, and is accessible from the dropdown menu in the top-middle of the browser window. There, you can see how many features each layer has, toggle their visibility, or delete them.
- To export, go to Export in the upper-right corner, and select a desired file format.
The choice of export formats is shown in Figure 11.15.
As of July 2020, these are Shapefile, GeoJSON,
TopoJSON (similar to GeoJSON, but with topographical data), JSON records, CSV,
or SVG (Scalable Vector Graphics, for web and print). If you export more than one layer at a time,
Mapshaper will archive them first, and you will download an
output.zipthat contains all exported layers.
Tip: Mapshaper doesn’t work with KML or KMZ files, but you can use GeoJson.io to convert these.
Edit data for specific polygons
You can edit attribute data of individual polygons (and also points and lines) in Mapshaper. Figure 11.16 shows you how.
- Import the file whose polygon attributes you want to edit.
- Under the cursor tool, select edit attributes.
- Click on the polygon you want to edit. A pop-up will appear in the upper-left corner listing all attributes and values of the polygon.
- Click on any value (underlined, in blue) and edit it.
- When you are done, export your geospatial file by clicking Export and choosing the desired file format.
Simplify map boundaries to reduce file size
You may not need precise and detailed map boundaries for data visualization projects where zoomed-out geographies are shown. Detailed boundaries are heavy, and may slow down your web maps.
Consider two maps of the contiguous US states, also known as Lower-48, shown in Figure 11.17. The map on the left is more detailed and is about 230 kilobytes, but the map on the right is only 37 kilobytes, 6 times smaller!
- Import your geo file to Mapshaper. You can use the sample contiguous US states GeoJSON.
- Click the Simplify button in the upper-right corner. The Simplification menu will appear, where you can choose one of three methods. We recommend checking prevent shape removal, and leaving the default Visvalingam / weighted area. Click Apply.
- You will see a slider with
100%appear on top (Figure 11.18), replacing the layer selection dropdown. Move the slider to the right and see the map simplify its shape as you go. Stop when you think the map looks appropriate (when the shapes are still recognizable).
- Mapshaper may suggest to repair line intersections in the upper-left corner. Click Repair.
- You can now export your file using the Export feature.
Tip: You may find the US shape a bit unusual and vertically “shrunk”.
In Console, type
-proj EPSG:3857 to change projection to Web Mercator,
which is more common.
Dissolve internal polygons to create an outline map
Mapshaper’s most powerful tools are available through Console, which allows you to type commands for common map editing tasks. One of such tasks is to create an outline map by removing the internal boundaries. For example, you can dissolve state boundaries of the US map in the previous exercise to get the outline of the country, like is shown in Figure 11.19.
Click the Console button, which opens a window to type in commands. Enter the command below, then press return (Enter).
You will see that internal boundaries became lighter color, and that’s Mapshaper’s way of saying they no longer exist. You can now export your outline shape.
Clip a map to match an outline layer
The state of Connecticut consists of 8 counties, which in turn are divided into towns. There are a total of 169 towns in Connecticut. Imagine you are given a boundary file of all 169 towns, and the outline of Hartford county. You need to “cut” the original towns map to only include those towns that fall within Hartford county.
Mapshaper allows you to do just that using one simple
- Import two boundary files into Mapshaper. One is the larger one that is being clipped (if you use sample files, ct-towns), and one is the desired final shape (hartfordcounty-outline). The latter is what ArcGIS calls the “clip feature”.
- Make sure your active layer is set to the map you are clipping (ct-towns).
- In the Console, type
-clipfollowed by the name of your clip layer, like that:
- You should see your active layer got clipped. Sometimes you end up with tiny “slivers” of clipped areas that remain alongside the borders.
If that is the case, use the
-filter-sliverscommand to remove them, like that:
-clip hartfordcounty-outline -filter-slivers
- Your Mapshaper state should look like pictured in Figure 11.20. You can now save the file on your computer using the Export button.
Remove unwanted data fields
Sometimes map features, such as polygons, lines, and points, contain unwanted attributes (or fields, or columns) that you may want to remove.
In Console, use
-filter-fields editing command to remove unnecessary fields.
For example, remove all fields except town:
If you want to leave more than one field, separate them by a comma, but without spaces, like that:
Note: If you leave a space after comma, you will get a Command expects a single value error.
Join spreadsheet data with polygon map
Combining spreadsheet data with geographical boundaries is a common task for geospatial practitioners. Imagine you have a file with Connecticut town boundaries, and you want to add population data to each of them in order to build a choropleth map.
Mapshaper provides a powerful
-join command to join such files. Remember that you need some common keys
in both datasets (such as town name, or state, or country) in order to join files. Otherwise Mapshaper has no way of knowing which
numbers belong to which polygons.
- Import both geospatial file and a CSV dataset into Mapshaper using Quick import box.
- Make sure both files appear in the drop-down list of layers. Your CSV data will be shown as something that resembles a table. Use the Cursor > inspect attributes tool to make sure the data is imported correctly. If you use the sample CT files, note that the ct-towns layer has name attribute with the name of the town, and ct-towns-popdensity has town names in the town column.
- Make your geospatial layer (ct-towns) is the one active.
- Open Console, and use the
-joincommand, like that:
-join ct-towns-popdensity keys=name,town
ct-towns-popdensity is the CSV layer you are merging with, and
keys are the attributes that contain values to join by.
In case with our sample files, these would be town names which are stored in
name attribute of the map file, and
of the CSV file.
- You will see a message in the console notifying you if join was performed successfully, or if Mapshaper encountered any errors.
- Use the Cursor > inspect attributes tool to make sure you see CSV columns as fields of your polygons, like is shown in Figure 11.21.
- You can now save the file to your computer by clicking the Export button.
Tip: To avoid confusion, it may be useful to re-name your CSV column that contains key values to match the key attribute name of your map.
In our example, you would rename town column to name column in the CSV, and your command would end with
Do you remember aggregating address-level point records of hospitals into hospital counts per state? Now is a good time to find that .CSV file and practice your merging skills.
Count points in polygons with Mapshaper
Mapshaper lets you count points in polygons, and record that number in polygon attributes using
Make sure your polygons (not points) layer is active by selecting it from the dropdown menu.
In Console, perform
count()function, like that:
-join hospitals-points calc='hospitals = count()' fields=
This command tells Mapshaper to count points inside hospitals-points layer and record them as hospitals attribute of the polygons.
fields= part tells Mapshaper to not copy any fields from the points, because we are performing many-to-one matching (many hospitals
per state, in our case).
Use the Cursor > inspect attributes tool to make sure polygons obtained a new field with the recorded count of points, like is shown in Figure 11.22.
Save the new file using Export button and choosing the desired output format.
More about joins
As you saw in counting points in polygons example, you do not need to specify keys if you want to perform join based on geographical locations between two geo files. If one of your files is a CSV, you need keys.
If you don’t have a CSV table that matches the columns in your boundary map data, you can easily create one. Upload the boundary map to Mapshaper, and export in CSV format. Open the downloaded file in any spreadsheet tool. To match data columns in the CSV spreadsheet, use the VLOOKUP function.
In real life, you will rarely have perfect files with one-to-one matches, so you might want to have more information about which features didn’t get matched so that you can fix your data. Mapshaper helps you keep track of data that is not properly joined or matched. For example, if the polygon map contains 169 features (one for each town in Connecticut), but the CSV table contains only 168 rows of data, Mapshaper will join all of those with matching keys, and then display this message:
[join] Joined data from 168 source records to 168 target records [join] 1/169 target records received no data [join] 1/169 source records could not be joined
To get more details on which values were not joined, add
unjoined unmatched -info flags to your join command, like this:
-join ct-towns-popdensity keys=name,town unjoined unmatched -info
unjoined flag saves a copy of each unjoined record from the source table into another layer named unjoined.
unmatched flag saves a copy of each unmatched record from the target table to a new layer named unmatched.
-info flag outputs some additional information about the joining procedure to the console.
Merge selected polygons with join and dissolve commands
In Mapshaper, you can merge selected polygons into larger “clusters” using
Imagine that you are employed by the CT Department of Public Health, and your task is to divide 169 towns into 20 so-called Health Districts and produce a new geospatial file. By the way, health districts are a real thing in Connecticut.
You should begin by creating a crosswalk of towns and their health districts. It can be as simple as a two-column CSV list of a town and its district. Because your boss didn’t give you a list of towns in a spreadsheet format, but instead a GeoJSON file with town boundaries, let’s extract a list of towns from it.
- Import ct-towns.geojson to Mapshaper using Quick import box.
- You can use the Cursor > inspect attributes tool to see that each polygon has a name attribute with the name of the town.
- Save attribute data as a CSV file using Export button. Open the file in any spreadsheet tool. You will see that your data is a one-column file with a *name& column that lists 169 towns.
- Create a second column titled merged and copy-paste values from the first, name column. At this point your spreadsheet contains two columns with the same values.
- Pick a few towns, for example West Hartford and Bloomfield, and assign “Bloomfield-West Hartford” to their merged column, like is shown in Figure 11.23. You may stop right here and move to the next step, or keep assigning district names to a few other neighboring towns.
- Save this new file as ct-towns-merged.csv, and drag-and-drop it to Mapshaper on top of your ct-towns layer. Click Import.
- This new CSV layer will be added as ct-towns-merged and will appear as a series of table cells. From the dropdown menu, select ct-towns to get back to your map.
- Now you are ready to merge certain towns into districts according to your uploaded CSV file. Open Console, and type:
-join ct-towns-merged keys=name,name
to join the CSV layer with the boundaries layer that you see on the screen, followed by
to dissolve polygons of towns according to the merged column of the CSV file.
In our example, only Bloomfield and West Hartford are dissolved into a combined “Bloomfield-West Hartford” regional health district (with the shared boundary between towns becoming grayed out), and all of the other polygons remain the same. Figure 11.24 shows the final result.
You can inspect attribute data of polygons using Cursor > inspect attributes tool, and save the resulting file using the Export button.
Learn more advanced MapShaper methods
TODO: Add Mapwarper section here