"CONTRACTING FOR GIS SERVICES: Part II - What you get."

The following sections provide brief summaries, in outline form, of the main points presented by the listed speakers at the February 2, 1999 meeting of the Massachusetts Geographic Information Council (MGIC). 

Moderator - Neil MacGaffey, GIS Administrator for the City of Newton

---

Overview

This meeting was the second of a two-part MGIC series on Contracting for GIS Services.  This session covered Part II - What you get. The January meeting covered Part I - What You Ask For. During this month's session we heard from speakers who are familiar with the receiving end of the GIS contracting relationship - "What You Get." The speakers are people who know how to get what they asked for from consultants. They discussed the Quality Assessment (QA)-Quality Control (QC) process for the most common types of contracted GIS services.

---

Presentation by Michael Terner, Vice President, Applied Geographics, Inc.
"The Importance of Quality Control and Quality Assurance To Successful GIS Implementations"

Objectives of This Presentation

• Overview of GIS Implementation project flow
• General discussion of QA/QC
• What is it?
• Why is it important?
• Who does it?
• Hint at QA/QC tasks and issues at major junctures of GIS project
• Other speakers will cover specifics
• Provide some advice

Overview of GIS Development Process
 - QA/QC is not just for data

 - View Image of GIS Development Process (11 kb)

What is QA/QC?

• QA/QC = Quality Assurance & Quality Control
• Functional Definition
• Process of ensuring you get what you pay for
• High quality products
• Products that match your specification/expectation
• Errors are identified and addressed before acceptance
• Procedural Definition
• A series of specifics checks and procedures to evaluate the quality of products
• Other speakers will address these specifics

Why is QA/QC important?

• You deserve good products
• Good planning, good data, and good applications
• Bad products can undermine credibility
• Many GIS products - especially data - form a foundation for subsequent products
• Your vendors are “on the hook” to fix problems that you identify during a contract
• Examining data helps you understand the data better
• Identify problems while you still have leverage

Who does the QA/QC?

• City/Town must be involved
• You know your terrain
• Best group for “data content” issues during checkplot review
• More cost effective for field work
• Role for consultants?
• Technical field
• You may have never tackled a project like this
• Flyover project can precede internal GIS staffing
• Checks for digital data
• Conformance to database design
• Topological integrity (labeling, pseudo-nodes, etc.)
• Topological review of checkplots
• Feature overlap issues
• Attribute coding issues
• Support at meetings: articulate issues, barter compromises
• Potential scenarios for QA/QC
• Scenarios 1a and 1b
• Community does everything
• Consultant does everything
• Scenario 2
• Community reviews checkplots for data content
• Consultant reviews digital data for database design issues and checkplots for topological issues
• Scenario 3
• Consultant works with community to develop QA/QC plan
• Consultant provides some training in QA/QC and/or electronic tools
• Community executes the plan

Some QA/QC advice

• A Good Specification is Critical
• Specification should be made clear in the procurement (RFP)
• Specification is basis for QA/QC
• For data products, specification = Database Design
• Tough to make someone do something that is not in the specification
• Specify that vendor should provide/explain internal QA/QC that they perform prior to product delivery
• Perform good QA/QC on your Needs Assessment
• Read the document thoroughly
• Don’t skip to the spreadsheet
• There are plenty of mediocre/bad Needs Assessments out there
• Are recommendations feasible and realistic? 
• Are recommendations in tune with fiscal climate?
• Is plan balanced covering hardware, software, data and staff?
• Does plan address needs/issues of all stake-holders?
• Does plan have enough detail to answer questions you’ll get during funding battles?
• Make us earn our money

Conclusions

• QA/QC is a critical component for GIS project success
• QA/QC will cost you money in either time and/or contracting
• QA/QC will save you money
• Future costs of fixing problems
• Maximizing value of existing dollars spent
• Recognize that QA/QC must be pursued during all phases of GIS development
• It’s not just for data

---

Contact Information:

Michael Terner Senior Vice President of Municipal Services Applied Geographics, Inc.  100 Franklin St. Boston, MA 02110 Phone: (617) 292-7642  Email: mgt@appgeo.com Website: http://www.appgeo.com

---

Presentation by Michael Trust, GIS Analyst, MassGIS
"QA of Digital Orthophotos and Topographic Contours"

The Vendor

• Intermap Technologies Nepean, Ontario, Canada 
• produce Digital Elevation Models (DEM’s) and associated value added products such as digital maps and Ortho-Rectified Images (ORI’s).  
• Contract began in 1994; continues through June, 2000 

The Products 

• Digital Orthophotos 
• half-meter pixel resolution 
• .bil format 
• grayscale shades range from 30 to 225 
• 8000 x 8000 pixels (4km. X 4km.) 
• film scanned at 15 microns 
• each image is 64 megabytes 
• meet or exceed National Map Accuracy Standards to the extent that 90% of well-defined features fall within 0.5mm of their true position on the ground at the nominal output scale of 1:5000 (2.5m on the ground) 
• Maximum displacement of well-defined features is less than 5 meters 
• Digital Terrain Models (DTMs)
• Points collected on analytical stereoplotters at sufficient density to support generation of  3-meter contours conforming to National Map Accuracy Standards (+ or - 1.5 meters) 
• Scan lines about 75m apart - variable density 
• Breaklines along significant linear features collected -- with distinctions made between “hard” and “soft” breaklines 
• DTM format Sample record:

249051.880 917979.791     20.593  SOFT 2
249051.907 917979.791     20.621  SOFT 4
249475.978 917978.316      0.852  HARD 1
249481.873 917977.407      0.852  HARD 2
Breakline information built in to DTM (HARD, SOFT, and numeric code)

View image of Example of Point Compilation (5 kb), variable density: (point compilation is denser along breaklines)
 

• Road Centerlines 
• interpreted atop orthophotos during DTM compilation 
• all public roads visible on photo, some accessways and railroads 
• MHD Roads inventory eventually will be attached to 5k linework 

MassGIS Processing 
We do some work ourselves:

• Resample orthophotos to 1-, 2-, and 5-meter resolution tiffs in ARC/INFO GRID module 
• Generate 3-meter contours, point elevation coverages, and breakline coverages from DTMs using ARC AMLs 
• use CREATETIN and TINCONTOUR for creation of contours 
• All 5k data tiled by Orthophoto Index (View Image of OQ Index - 6kb) 

The QA Process 

• Orthophotos QA done in conjunction with QA of contours - Why duplicate QA time? -- the orthos are crucial to the QA of the contours
• Resampled orthos QA’d with contours 
• Half-meter orthos QA’d with road centerlines 
• Orthophotos 
• How do they look? 
• Properly clipped and registered? 
• Edgematched with adjacent tiles? 
• Brightness/contrast consistent? 
• Hairs, scratches, or other unwanted marks visible? 
• Does resolution match the specs? 
• We decided on no tonal blending between adjacent sheets in coastal areas. 
• Positional accuracy not tested by MassGIS
• A pilot project to test the accuracy of known survey points was done at the beginning of the project. The result was that Intermap surpassed our expectations. 
• For every area controlled by Intermap, an independent surveyor randomly checks accuracy of control points and certifies every map sheet 
• Based on stamp of approval by surveyors, the orthos are considered a 1:5k “base map” for the state - other datalayers will be aligned to photos 
• open space 
• zoning 
• wetlands and streams 
• land use 
• Out of 627 ortho tiles, only one needed to be redone - a 99.8% success rate - the tile was clipped improperly 
• problem discovered when contours were overlaid during QA - the DTM was ok, but the image was not 
• 3-meter Contours 
• Overlay contours, breaklines and points on 1-meter orthos in menu-driven Arcedit session  
• Even though we produce the contours, checking them reveals the quality and accuracy of the DTMs produced by Intermap -->we are really QA-ing the DTMs (looking at hundreds of text files is meaningless!!) 
• Are contours, breaklines and points complete for the entire tile? 
• Do contours follow the “lay of the land”? 
• Are contours broken? Do they intersect? 
• Do contours cross “hard breaklines” and go into water bodies or into the ocean?
• Are “hard” breaklines for shorelines of equal elevation and non-broken? 
• How do contours behave around bridges and highway overpasses? 
• 3-meter Contour Recurring Problems: 
• lines in the water 
• in ocean 
• in rivers around bridges 
• “cones and strays" 
• “spikes” 
• “dense blobs” - thick masses of contours that don’t follow terrain 
• The culprit: Something’s wrong with the DTM! 
• miscoded point 
• extra point 
• part of DTM record is missing 
• The solution: 
• record the coordinates of the error 
• fix it myself (if I can) 
• alert Intermap for redo 
• In most cases I have Intermap re-visit the DTM(s) in question, providing them a detailed description of the problem. 
• If I fix the problem I send the corrected DTM back to Intermap so they can archive the file. I tend to have Intermap do the fix - it’s in the contract. 
• Once Intermap delivers the corrected DTMs, I review them all over again 
• review my notes for the specific errors 
• check for any new problems introduced in the “fixing” 
• send them back if still not right 
• The contours are not ok until I say they’re ok - not afraid to keep going back to Intermap until they are problem-free. 
• We have a good relationship with the project manager, who understands the technical issues and makes correcting these problems a priority 
• Intermap now watches for these issues more closely and promises that future deliveries of DTMs will be problem-free. 
• Of recent batches, about 20% of DTMs had some problem that needed more attention by Intermap 

Conclusions

• An understanding of the nature of the problems and how the data is developed is helpful, if not necessary, in performing proper QA for any data set. 
• you need to “know what to look for” so that you can determine that you got what you paid for 
• Automation of QA process helps speed things along 
• Careful and complete documentation of any error is essential in correction 
• A good relationship with the vendor results in prompt and complete fixes of problems

---

Contact Information:

Michael Trust  GIS Analyst  MassGIS  20 Somerset St., 3rd Floor  Boston, MA 02108  Phone: (617) 727-5227 ext. 322 Fax: (617) 227-7045 Email: michael.trust@state.ma.us Website: http://www.state.ma.us/mgis

---

Presentation by Peter Bujwid, GIS Manager, City of Cambridge
"Parcel Mapping QA/QC"

City of Cambridge

• Size - 7.13 sq. miles (.70 sq. miles of water) 
• Population - 99,890 folks   (1994) 
• Number of Land Parcels - 15,000 
• Number of Assessing Accounts - 22,500 
• 291 tax maps  
• Aerial Survey April 1995 
• 100 Scale Data using ASPRS Standards for 1” to 100’ 
• Includes Orthophotographs, Planimetric, Topographic, Tax, Sewer and Water Data Layers 

Parcel Mapping in Cambridge

• 1995   Scrubbing of source documents 
• Time:  January 1996 - January 1997 
• Cost:   $116,227.79 based on $7.63 per physical/legal   lot for 15,233 lots 
• Approach: 
• Prepare for delivery 
• Pilot cycled 3 times 
• Phased delivery of data including hard copy check plots and digital data files by individual map sheet 
• In-house QA/QC 

Preparing for QA/QC

• Defining the QA/QC approach 
• Complete QA/QC or sample based approach 
• Map and database review 
• How will QC proceed, what are the mark up procedures  
• Prepare the QA/QC team 
• Who will be on the team 
• What is the time commitment 
• Review QC process 
• Defining an acceptable product 
• An quantitative approach 
• Acceptance period 
• Multiple source conflict resolution 
• Scrubbing 
• Translucent vellum copies. 
• Preparing the consumer 
• What was converted (Parcel maps not deed research) 
• How can it be used 
• How can it not be used 

QA/QC - A three stepped approach 

• Step 1: Paper document review 
• Line-work review 
• Symbology review 
• Annotation review 
• Missing map lots 
• Missing or incorrect parcel dimension annotation 
• Parcel size (sq. ft.) annotation 
• Miscellaneous text 
• Text position 
• The Call Sheet 
• QC Call 
• Step 2: Digital data review 
• In final Arc/Info format 
• Automated AML based QC programs 
• Conversion company QC reports 
• Consultant 
• City 
• Fuzzy, dangle, items, definitions, content, valid values, bnd values 
• Edge matching  
• Step 3: ArcView review 
• Consultant Arc/Info based QC menu system 
• Cambridge use of ArcView 
• Visual edge match 
• Color coding  
• Check map-lot 

The Results

• View Image of "Before" (15 kb) - The original paper product 
• View Image of "After" (99 kb) - The GIS-produced parcel map 

Conclusions

• Clarify acceptance criteria 
• Use the pilot to refine the product 
• Develop a realistic timetable 
• Use QC as a springboard 
• Multi-participant process: vendor, Project Manager, QC Team, consumer 

---

Contact Information:

Peter Bujwid GIS Manager City of Cambridge Phone: (617) 349-4140

---

Presentation by Michael Doyle, Network & Information Systems, Town Of Wellesley
"QA of Planimetric Data"

Introduction

• Planimetric data is the foundation of municipal GIS
• Thorough QA/QC of this is crucial
• Timely QA/QC is also crucial

Pilot Area

• Very Important
• Work out all bugs
• Let contractor get a feel for your desires
• Use as many revisions as needed to resolve all problems

The QA process

• Automated procedures
• On screen querying, inspection
• Hard copy inspection
• Visual Inspection

Automated Procedures

• Program written to check attributes:
• Valid information according to plans
• Everything is coded
• Coverage is properly cleaned
• Prints out a report for you

On Screen Querying/ Inspection

• Theme on theme selections for overlaps
• Visual inspection of coding
• Check for feature coincidence

Hard Copy Inspection

• Feature coincidence
• Coding
• Overlay onto existing data (old plans) for thoroughness

Visual Inspection

• In the field
• This is the only real way to ensure accurate data
• Takes a lot of time.

Familiarity of Area

• I grew up in Wellesley, but if you can get someone who is familiar with infrastructure that will help.

Is the QA complete?

• Yes, as far as the contractor is concerned.
• Realistically, no.
• GPS survey is checking data.
• Many of the current data development apps in town use this data, and is therofre being checked.

---

Contact Information:

Michael Doyle  Network & Information Systems Town of Wellesley Phone: (617)431-1019 ext. 280 E-mail: nis@ci.wellesley.ma.us Website: http://www.ci.wellesley.ma.us/nis/index.html

Last Updated 4/19/2000
EOEA Disclaimer | Privacy Policy