Community Access to Natural Resources Information   

...the power of shared information

DIY Home

1:About the DIY

2: CANRI at a Glance

3: Getting started

4: Answer Finder

5: Technical View

6: Applications

7: Managing Metadata

8: Data Serving

9: Testing

Contacts and Support

Glossary

8 Guide to Data Serving

Who should read this Guide

This document is intended for technical users who intend to serve their own data, ie, you will be concerned with establishing CANRI-compliant online data services.

If you do not intend to provide online access to your data products, simply follow the procedures for registering with the NRDD at MET Online metadata management. This process ensures that users of the NRDD can see how you wish to make the data product available.

If you are particularly interested in displaying and interacting with data, please refer to Applications: Solutions for your end-users.

Figure 12: Orientation to Data Serving

Software and Solutions

This section documents CANRI-specific tunings, or adjustments, that are associated with various software packages. The listings are in alphabetical order by name of package. Where possible, we have included summary procedures supplied by the software developers.

Note: If you are a technology supplier with relevant solutions not listed in this section, please get in touch with CANRI so we can organise a heading for you.

Table 14: Summary of References: Data serving Solutions

8.1 Quick start

Here's an overview of the data serving process from start to finish. This a slightly re-worded version of the CANRI Four Step (repeated here in case someone lifted the first chapters from your copy of the DIY).

Prepare data for CANRI

We can help you do a lot of tricky tech with CANRI, but we can't invent data. Organising your data products for access through public applications may involve several rounds of investigation and prototyping. Application designers and web managers will want to know what's possible within the CANRI framework, so this section will help you answer specific questions about formats and capabilities available. More tips related to data product design are provided at: Suggested steps in solution design.

Establish a web environment and data server

If you don't have an existing technical infrastructure, you'll need to choose a serving environment and GIS platform suitable for your needs.

The most popular open-source solution uses Apache running on Linux with TomCat or Resin as the Java servlet container. A database and server are needed: MapServer 3.5 is a good choice, with GeoServer offering specialty shape functions if you need the sort of sophistication available in PostGIS. If you're serving data, you can use JDBC connections to a range of applications, such as MySQL, MSAccess, and MSExcel using an OpenGIS WFS connector such as WFSLite (licence fee applies).

There are a number of commercial offerings for these functions, suited to a range of budgets. The product summary references in this section provide links for more information.

If you already have a data serving environment set up and running, then you need to refer to the specific package references for more details on setting up your environment.

Hosting services

If data serving is not yet a possibility, you might try to organise a hosting arrangement with a CANRI partner who already provides OpenGIS-compliant servers.

For example, DLWC have provided, upon negotiation, data hosting for CANRI agencies. To discuss the particulars of such an agreement suitable for your agency, please contact the CANRI Program Director.

Register data and service metadata

Once you have your data server set up and tested, you need to register the server to the CANRI catalog. This metadata will include information that instructs CANRI applications how to access the data layer offered at the service. The CANRI Metadata Managers Guide contains all the detailed procedures required for this step.

Note: Until your service and your map layers are registered with the CANRI Catalog (NRDD + services), your data will not be available to CANRI applications.

Test with a viewing application

The easiest way to view your data online is to use the NRA at http://www.canri.nsw.gov.au/atlas/. More details about applications you can build yourself is provided under Applications: Solutions for your end-users

If for some reason you don't have a CANRI login, or want to simply test your map server with an OpenGIS client, then CubeWerx's WMS interface (cubeview) can be useful to test a new server.

Cubeview is located at: www.cubewerx.com/demo/cubeview/cubeview.cgi

Enter your server's URL at the bottom of the page and click "GO". The default set of layers in the interface should be replaced with your server's layers.

8.2 Evaluating your options

For many organisations, technical policy and existing systems will mean there are only a few practical options. You pretty much will need to work with what you have. To get a better idea of what's possible with your existing set-up, browse through the summary reference tables for your installed packages.

If you're starting from scratch, you've got a bit of study ahead of you. Providing extensive details on each possible combination of hosting environment, dataserver, application, and data product is beyond the scope of this manual. If you require information in addition to what we have provided here, your first option would be the CANRI-Talk mailing list. After that, you may wish to contact a qualified contractor to evaluate your options in detail.

In the following table, we provide some of the typical evaluation criteria you can use to compare the features provided by different combinations of products and solutions.

Table 15: Evaluation criteria for data serving

There is a list of all products with conformance to OpenGIS implementation specifications at: http://www.opengis.org/cgi-bin/implement.pl You can filter this listing for your desired standard, developer, or component type.

Note: OpenGIS conformance does not necessarily equate to CANRI interoperability. For more information about checking your product's suitability for CANRI, see the summary references for that product, and also note Testing: Conformance and Interoperability in this document.

8.3 Organising your Data

This section looks more closely at the specifics of data formats. Many of these issues are generic to the production of digital maps. The information provided here is summarised for use in the CANRI context.

Map layers

An important consideration for map layers is both the originating format, and the capabilities of the dataserver and middleware to be used by requesting applications.

Map layers may be photographic, single/tiled vector, raster, single/tiled image etc. If you'll be using the existing CANRI middleware product (MapBroker), you must ensure that your dataserver can deliver:

• GIF: 87 or 89a: GIF files must be of a known geographical extent, projection and scale, as described in the corresponding metadata record.
• Imagemap: The WMS and WFS specifications allow server-side imagemap formats with coordinates in real world units (in supported projections).
• PNG: Portable Network Graphics
• JPEG: Note that there is no transparency to JPEG images; these layers will obscure all lower layers in a combined display.
• SVG: SVG is a rendering format for XML data. It can be supported at the client side by SVG-capable applications.

Geo-referenced data

The OpenGIS WFS protocol allows you to establish a transaction connection to virtually any ordered information collection format, including:

• Delimited text (ASCII)
• Other (non-webmap: CORBA, SQL, etc.)
• Simple Features
• Grid Coverages
• Coordinate Transformation Service
• Shapefiles

The specific features supported by each WFS connector will need to be evaluated in detail within the context of your operating environment.

Spatial coordinates: Common Projections and Datum

The common projection methods for NSW-wide and Australia-wide maps produce various distortion effects. In considering the most appropriate projections for a given data product, it may be important to refer to this table of recommendations.

Despite the distortion observed in Plate Carée projections, it is used for the default display for most scales. One reason for this was that the Australian Coastal Atlas had settled on this choice. Another was that the Airview raster base data to be supplied by NSW Surveyor-General's Department will be in Plate Carée.

Policy note on GDA: Even though its real-world adoption has been slow (in part because it is almost the same as WGS84), the use of GDA is has been an official requirement of the NSW Cabinet for about five years (see Cabinet Minute (#51) 264-98 of 1998).

Table 16: Recommended Projections and Datum

Notes:

• Plate Carée is a map projection in which parallels of latitude and meridians of longitude are displayed as a square grid.
• Universal Transverse Mercator (UTM): Note that the Australian Map Grid (AMG) is UTM projection with AGD66 datum.
• Maximum shift between datums in NSW is approximately 200m, which is not discernible when displayed at scales smaller than 700K. So any datum (AGD66, WGS84, GDA) may be used at these scales. The WGS84 and GDA datums are equivalent at scales to 1:500.

8.4 ArcIMS / WMS and WFS Connectors

Summary of References 10: ArcIMS (WMS and WFS)

Overview

ArcIMS is the web server that connects with the ESRI suite of GIS spatial products.

How ArcIMS is used

Although ArcIMS is primarily intended for use with ESRI applications, an OpenGIS WMS connector is provided, and the company has expressed its commitment to providing WFS and web services (catalog) support as the OpenGIS protocols stabilise.

Procedures

These procedures have been supplied by ESRIAU. For more information and the latest updates, please use the contacts summarized above.

Install WMS components

The first thing is to install all the WMS components provided with ArcIMS.

This can be done when installing ArcIMS, or after ArcIMS has already been installed, the process is the same either way.

Choose "custom installation" from the ArcIMS installation options, then choose "options" from the samples menu and add the WMS connector samples.

For more information, see the ArcIMS Installation Guide, Step 3: Install ArcIMS, Installing ArcIMS custom Application Server Connectors for your platform for instructions on how to configure ArcIMS with WMS.

Once ArcIMS is installed, the WMS connector is installed in the same location as the Servlet connector although it is disabled by default.

ie. <tomcat installation directory>\webapps\ROOT\WEB-INF\classes

Configure for WMS requests

To enable ArcIMS to receive WMS requests, edit the WMSEsrimap_prop file at:

<tomcat installation directory>\webapps\ROOT\WEB-INF\classes\WMSEsrimap_prop

and change these lines:

Produce capabilities document

The Capabilities request to the WMS connector on a MapService (a map layer, eg. Dubbo) first looks for a user defined capabilities file called "Dubbo.XML" in the capabilities directory.

If the file does not exist, the capabilities are generated dynamically, but an exception "The system cannot find the file specified" is written to the log file. This exception can be ignored as the response to the request was created dynamically.

Full documentation on each of the properties and available WMS requests can be found in the ArcIMS help:

<ArcIMS installation directory>\Manager\help (search for WMS or open the WMS section under contents)

Restart Java servlet engine and test

Once you have changed these properties, restart the servlet engine, eg. Tomcat .

Then test the following requests in a web browser (the responses that you should receive from ArcIMS 3.1 are listed below the requests)

http://<web server name>/servlet/com.esri.wms.Esrimap?CMD=connectorping

ArcIMS WMS-OGC Connector V 3.1

Build_Number=435.1160

Version=3.1

Enabled=true

http://<web server name>/servlet/com.esri.wms.Esrimap?CMD=ping

ArcIMS WMS-OGC Connector Version 3.1

Running at <webserver name>:80

View WMS layers

Copy the WMS samples from <ArcIMS Installation directory>\Samples\WMS to your website folder and follow the instructions in the readme file to set up the map services required.

The DHTML_wmtclient has the ability to generate a capabilities document, as well as pan zoom and identify functionality.

For more specific information, the ArcIMS forums have a WMS section http://forums.esri.com/forums.asp?c=64#784

(It is also possible to search this section to retrieve archived posts if they don't appear on the front page.)

ArcOnline has updates and troubleshooting information http://arconline.esri.com/

Extending ArcIMS: WFS, scripts

ArcScripts (http://arcscripts.esri.com/) also have ArcIMS related downloads, such as "Including OpenGIS data sources in the htmlviewer", and http://arcscripts.esri.com/details.asp?dbid=11981

There is a beta WFS connector and updates to the WMS connector are available from http://mapster.esri.com/ogc/download/

8.5 Autodesk MapGuide

Summary of References 11: Autodesk MapGuide

MapGuide implements the OpenGIS WMS specification in its web server.

8.6 CubeSERV

Summary of References 12: CubeSERV

CubeSERV is a map server that provides access to the CubeSTOR^TM Spatial Warehouse. It is also fully OpenGIS compliant and therefore "cascades" to access images and data from other OpenGIS compliant map servers.

8.7 DSLite

Summary of References 13: DSLite

DSLite is prototype software developed under the original ICMISS program, prior to the formation of OpenGIS standards. CANRI policy is to adopt OpenGIS standards. At this stage there are no plans to upgrade DSLite to support OpenGIS standards as its functionality has been superceded by WFSLite, GeoServer, MapServer 3.5, and others.

For urgent matters of support on existing DSLite installations, please contact the CANRI Technical Officer for guidance.

8.8 Exposure Image / Spatial Server

Summary of References 14: Exposure Image / Spatial Server

Exposure offers an image server and spatial server with OpenGIS implementations:

• conforms to OpenGIS Consortium Web Map Server standards
• supports highly compressed ECW image files to save disk space
• achieves storage reduction of 20:1 without loss of quality

8.9 GenaMap / GenaWorld

Summary of References 15: GenaMap / GenaWorld

For those users still running GenaMap systems, there is a built-in function called WebRequester which provides basic WMS functionality. No FeatureInfo layer, no projections: returns images only.

Users

8.10 GeoMedia WebMap (Intergraph)

Summary of References 16: GeoMedia WebMap (Intergraph)

Intergraph offer a configurable application that demonstrates how to build an OpenGIS Web Map Server. The GeoMedia 5.0 product suite runs with the Oracle 9i Standard Edition

8.11 GeoServer

Summary of References 17: GeoServer

The GeoServer project is a Java (J2EE) implementation of the OpenGIS Consortium's Web Feature Server specification. It is free software, available under the GPL 2.0 license.

GeoServer requires PostGIS as its spatial database.

8.12 ER Mapper: Image Server

Summary of References 18: ER Mapper: Image Server

The ER Mapper image web server provides the capability to serve very large images to the web. An OGC WMS interface has been developed and is being improved by Digital Earth http://www.digitalearth.com.au/.

8.13 MapInfo

Summary of References 19: MapInfo

MapInfo is supporting the Open GIS consortium (OGC) for web enabling of its software products, MapExtreme and Encounter. Release dates for WMS and WFS compliance have yet to be announced.

8.14 MapServer (MapServ, ex ForNet)

ForNet MapServer is still in use at DLWC. For all new installations, see following listing.

8.15 MapServer 3.5 / 3.6

Summary of References 20: MapServer 3.5 / 3.6

Overview

MapServer is an open-source project hosted at the University of Minnesota. MapServer is not a fully-featured GIS, but its web interface does support OpenGIS WMS/WFS queries. It is a solid platform for basic functionality. In essence, this software is an open-source equivalent of a commercial WMS/WFS.

The 3.5 version has been tested against the WebMap Composer's OpenGIS WMS and WFS queries. There are a small number of tunings required for CANRI interoperation currently. For more information on these adjustments, please send a message to CANRI-Talk, or contact the CANRI Technical Officer.

The current version is 3.6. Documentation is expected soon.

MapServer optionally comes pre-compiled for Windows environments, but can be compiled and run from source on any platform with a C-compiler.

How MapServer might be used

Typically, MapServer is used as a web front end to proprietary GIS packages that run internal to an organisation's network.

For example, an agency with GenaSYS installed on their intranet might establish a public web server for the purpose of providing limited public access to some core datasets.

In order to make the data available to the CANRI framework, the GenaMap export files would be read by a MapServer installation running on the public web server.

8.16 WFSLite

Summary of References 21: WFSLite

Overview

WFSLite is a servlet that accepts WFS queries (plus services extension) and returns GML which may then be symbolized by appropriate middleware.

WFSLite supports a limited number of geometries, but an extensive range of transaction models and backend connectivity via JDBC.

Procedure for installation and testing

This procedure have been supplied by the developer. Please see the Summary of References table above for the most current online version.

Establish database access

1. Determine the type and version of database you are using i.e. MySQL, PostgreSQL and copy the required JDBC driver in the WEB-INF/lib directory. Currently the directory contains valid JDBC Driver's for PosgreSQL 7.0, PostgreSQL7.1 and MySQL.
2. Configure the servlet to speak to the database by editing the DBConnectionManager.properties file, located in WEB-INF/classes/csgi/DBConnectionManager.properties. Exmaples for PostgreSQL and MySQL are shown in these files. You will have to edit it appropriately for your database.

Setup Capabilities

The capabilities document is stored in wfs/xml/GetCapabilities1.0.0

1. Read the example
2. Change it to match your database structure
3. Ensure that it is called GetCapabilities1.0.0.xml as this is the default name the servlet uses to call the document.

Comments are contained inside the example to guide you through it

Describe FeatureType

All table names correspond to TYPENAME in WFS queries. Thus for each FeatureType in the capabilities document a corresponding XML schema must be created for it and this file must be stored under wfs/xml/ the current example is test.xsd

1. Read the example
2. Change it to mirror your table structure
3. Ensure that the name inside the capabilities document exactly matches the name of the file and correspondingly the file name.

Comments are contained inside the example test.xsd to guide you through it

Setup Tomcat context

Only perform if you have not dumped the wfs.war into the webapps directory)

1. Setup the context for the project under in tomcat in the server.xml file.
2. Restart Tomcat.

Testing

Testing can be performed by using the test JSP pages hyperlinked above.
Some additional comments:

• Each WFS is automatically configured to support GET and POST queries
• Additional configuration is required for any BESPOKE querying interfaces.
• Please contact terence@socialchange.net.au for any further support and comments on the installation documentation.

[ up to top ]

© 2002 NSW Government - Community Access to Natural Resources Information    canri@canri.nsw.gov.au