1 - Instantaneous Values

1.1 - Intantaneous Values Frequently Asked Questions

Are new services planned? If so what will they be?

A number of new web services are being considered. In 2014, the USGS hopes to deploy an OGC-compliant Sensor Observation Service. A water quality web service is also on the planning radar for 2015 or later. Reference list web services (returning metadata information, such as a list of counties for a state) will be introduced piecemeal as needed to support projects. All such changes will be announced via the USGS Water Data Notification Service External Link.

What new features are likely to be developed for this service?

The development of a new Sensor Observation Service (SOS) in 2014 will serve equivalent data in a more “geo-friendly” format. The same data will be available via the SOS as is served by this service, with data served in the WaterML 2 format. Support for Excel output are planned for FY13 or FY14 for this service.

Will WaterML 2.0 be supported?

Yes, it will be supported by a Sensor Observation Service to be deployed in 2014 or 2015 and also by this service in 2014.

Where can I find older historical instantaneous values?

Currently values are supported from October 1, 2007 forward. Most historical streamflow data is available through the Instantaneous Data Archive External Link. Other data may be available through an inquiry with the local USGS water science center by writing gs-w-xx_nwisweb_data_inquiries@usgs.gov where xx is the state postal code (such as “ny”) of interest.

1.2 - Instantaneous Values Service Details

Instantaneous Values Service

You can use this service to retrieve recent and historical values for streamflow as well as data for other regular time-series parameters served by the USGS. This service provides these USGS water data in Extensible Markup Language (XML), Javascript Object Notation (JSON) and the legacy RDB (tab-delimited) format currently available from the USGS Water Data for the Nation site . More media types will follow.

How the service works

• This is a REST-friendly service, which means it is URL accessible and can be run from a browser
• The service can return recent water readings for one or more sites in one request
• Data from October 1, 2007 to the present can be returned with one request. However, certain operational data that is not quality assured (this typically includes temperature and precipitation) is limited to 120 days or less by the local USGS water science center responsible for the data.

With thousands of sites monitored across the nation, and with the majority of these sites having measurements for more than one type of data, the amount of data available is very large. No one user is allowed to download all of the data with a single call. The service has consequently been designed and engineered to facilitate common mass queries, defaulting to returning a narrower set of data. You are encouraged to make your queries efficient too, mindful that many others need access to the data. Always specify the minimum amount of data you need in your request, using built in filters and date ranges to the maximum extent possible.

Testing the service

Probably the best way to learn how to use the service is to try it out!

The USGS provides this tool that allows you to generate syntactically correct calls to the service. You can even run the query live in your browser. The XML output may look somewhat strange if you are new to XML. When you have perfected your query you can copy and paste the URL into your application to get the precise data you need.

Test the service now

Enabling gzip compression

Typically data is downloaded as plain text via Hypertext Transfer Protocol (HTTP). However, gzip compression is supported by this service. Use of gzip compression may markedly speed up acquisition of data, particularly on large queries. It also is a more efficient use of USGS servers, so we appreciate when you are thoughtful enough to use it. Whether you can receive the data in gzip compressed formats depends on the capabilities of your client. Web browsers support gzip compression natively, but most regular users will use specialized utility programs like wget and curl to acquire data. If you can handle gzip compression, please place the following string into your HTTP request headers: Accept-Encoding: gzip, compress

curl and wget are typically used to download data. They may be configured to use gzip compression if the server will accept it. You may also explicitly have to tell it to use gzip compression. If so these tips should work:

• curl: try adding the argument: -H 'Accept-Encoding: gzip,deflate'
• wget: try adding the argument: --header="Accept-Encoding: gzip"

gzip files are typically returned as a file with a .gz file suffix unless you instruct your program to uncompress it. See the gzip man page for instructions on uncompressing .gz files.

Output

Please note that most recent data are marked provisional, so these data should be interpreted with caution as it is possible (although unlikely) to be incorrect. See the USGS Provisional Data Disclaimer page for more information.

When using format=waterml (the default format), data are returned in XML using the WaterML 1.1 schema . WaterML is a schema that has recently been adopted by the Open Geospatial Consortium . The crucial data are the instantaneous values, which can be found inside the <value> tag. Example:

<ns1:values>
  <ns1:value qualifiers="P" dateTime="2010-06-02T08:45:00.000-05:00">9520</ns1:value>
  <ns1:qualifier qualifierID="0" ns1:network="NWIS" ns1:vocabulary="uv_rmk_cd">
    <ns1:qualifierCode>P</ns1:qualifierCode>
    <ns1:qualifierDescription>Provisional data subject to revision.</ns1:qualifierDescription>
  </ns1:qualifier>
  <ns1:method methodID="0">
    <ns1:methodDescription>sensor::1</ns1:methodDescription>
  </ns1:method>
</ns1:values>

The attribute qualifiers="P" indicates the value is considered provisional by the USGS. Please also note that the value is associated with the variable node that precedes it. The variable node indicates the parameter that is being measured or calculated. In this case, both the <ns1:values> and <ns1:variable> nodes are nested within the <ns1:timeSeries> node. For a full list of qualifier codes, please consult this reference .

<ns1:variable ns1:oid="0">
  <ns1:variableCode network="NWIS" vocabulary="NWIS:UnitValues" default="true" variableID="0">00060</ns1:variableCode>
  <ns1:variableName>Streamflow, ft&#179;/s</ns1:variableName>
  <ns1:variableDescription>UNKNOWN</ns1:variableDescription>
  <ns1:valueType>00060</ns1:valueType>
  <ns1:unit>
    <ns1:unitCode>UNKNOWN</ns1:unitCode>
  </ns1:unit>
  <ns1:noDataValue>-999999.0</ns1:noDataValue>
</ns1:variable>

Putting it altogether, this means that for this site, a provisional streamflow measurement of 9,520 cubic feet per second was recorded at 08:45 AM EST on June 2, 2010.

With other output formats, the location of the data will depend on the syntax of the format. You will need to inspect the format to locate the relevant data.

Error codes

Since this system uses Hypertext Transfer Protocol (HTTP), any application errors are reported in the HTTP headers. This means that when writing applications, it is important to first examine the HTTP status code that is returned in the HTTP response. The application server will return the error code along with a message describing the error in the event there is a problem. Programmers should always check the HTTP response code and if not a 200 handle the response as an exception. Among the status codes you may see:

Using the Web Service with Adobe Flex or the Flex API

Adobe Flex requires our server have a crossdomain.xml file indicating those domains that can access our web service using Adobe Flex. We are adding these on a case by case basis. If you need to access the service using Adobe Flex or the Flex API, please contact us using this form and indicate the domain of the server that will access the service.

CORS Support

This service supports the Cross-Origin Resource Sharing (CORS) specification. CORS permits browser-based asynchronous access to the service even though content originates from a server different than the one serving the web page. Otherwise the browser’s security controls would not allow content to come from USGS servers. Most, but not all browsers, support CORS. Some frameworks, like jQuery through the Ajax crossDomain setting, support CORS automatically.

URL Format

The URL must always be in this format:

https://waterservices.usgs.gov/nwis/iv/?<arguments>

where <arguments> are one or more HTTP GET parameter names and values based on the information below.

Specifying the URL Arguments

You specify the content that goes in <arguments>.

• Each URL argument name is followed by an equal sign followed by one or more values for that argument. Where multiple values are allowed for the same argument, you can separate values with commas.
• Arguments are separated by ampersands (&)
• The order of the arguments in the URL does not matter.
• If a URL argument name does not match one of the names below, it is ignored.

Here is an example of a valid URL that should return data:

https://waterservices.usgs.gov/nwis/iv/?sites=01646500&parameterCd=00060,00065

URL argument names and argument values can be in upper, lower or mixed case. They will all be handled correctly. All of the following will yield the same result:

• https://waterservices.usgs.gov/nwis/iv/?stateCd=ny
• https://waterservices.usgs.gov/nwis/iv/?statecd=ny
• https://waterservices.usgs.gov/nwis/iv/?STATECD=ny
• https://waterservices.usgs.gov/nwis/iv/?stateCd=NY
• https://waterservices.usgs.gov/nwis/iv/?STATECD=NY
• https://waterservices.usgs.gov/nwis/iv/?stateCd=Ny

Argument conventions

The following conventions are used below to document argument values:

arg1=[ a {,x | ,y} | b | c,d,...]

• square brackets [] are used to show a set of possible choices, with | delineating exclusive choices. You must select one and only one of these choices.
• curved brackets {} are used to show optional elements. They also may be delineated with | indicating exclusive choices. If used, you may select one and only one of these choices.
• ... indicates more than item may be specified if items are delineated by commas. Note the limitation on the maximum number of argument values allowed below.

In the above example, these would be the allowed legal values:

• arg1=a
• arg1=a,x
• arg1=a,y
• arg1=b
• arg1=c
• arg1=c,d,e,f

Major Filters

Single Site Queries

Want to only query one site? Use site (or sites) as your major filter, and put only one site number in the list! Example:

https://waterservices.usgs.gov/nwis/iv/?site=01646500

Multiple Site Queries

• Every query requires a major filter. Pick the major filter (sites, stateCd, huc, bBox, counties) that best retrieves the data for the sites that you are interested in.
• You can have only one major filter per query. If you specify more than one major filter, you will get an error.

Outputs

Format

Indent (indent)

Specifying Date Ranges

Data going back as far as October 1, 2007 are available for each site. You typically do not need all this data, so you generally express a range of dates needed. Here is how to limit the amount of values you receive:

Minor Filters

Additional filters can be applied after specifying a major filter. This further reduces the set of expected results. Users are encouraged to use minor filters because it allows more efficient use of this service.

Parameter Code (parameterCd)

Site Type (siteType)

Site was modified since (modifiedSince)

Agency Code (agencyCd)

Site Status (siteStatus)

Altitude (altMin and altMax)

Surface water filters

Drainage Area (drainAreaMin and drainAreaMax)

Groundwater Filters

Aquifer Code (aquiferCd)

Local Aquifer Code (localAquiferCd)

Well Depth (wellDepthMin and wellDepthMax)

Hole Depth (holeDepthMin and holeDepthMax)

Feedback

Please provide any feedback you have on this service using this form .

2 - Daily Values

2.1 - Daily Values Frequently Asked Questions

What new features are planned for the daily values service?

Likely new features include: Excel 2007 output, Google Earth and Google Map (KML) output, WaterML 2.0 output and search by site name.

Will WaterML 2.0 be supported?

The USGS hopes to support an OGC-compliant Sensor Observation Service in 2014 or 2015, which will be able to serve daily values.

2.2 - Daily Values Service Details

You can use this service to retrieve daily statistical data about the hundreds of thousands of hydrologic sites served by the USGS. This service provides USGS water data in WaterML 1.1 (an XML schema) as well as the legacy RDB (tab-delimited) format and a JSON (Javascript-friendly) format.

How the service works

• This is a REST-friendly service, which means it is URL accessible and can be run from a browser
• The service can return daily values for one or more sites in one request
• Data for historical, as well as currently active sites are available.

With hundreds of thousands of qualifying sites across the nation, the amount of data available is very large. No one user is allowed to download all of the data with a single call. The service has consequently been engineered to facilitate common mass queries, defaulting to returning a narrower set of data. You are encouraged to make your queries efficient too, mindful that many others need access to the data at the same time. Always specify the minimum amount of data you need in your request, using built in filters and date ranges to the maximum extent possible.

Testing the service

Probably the best way to learn how to use the service is to try it out!

The USGS provides this tool that allows you to generate syntactically correct calls to the service. You can even run the query live in your browser. The XML output may look somewhat strange if you are new to XML. When you have perfected your query you can copy and paste the URL into your application to get the precise data you need.

Test the service now

Enabling gzip compression

Typically data is downloaded as plain text via Hypertext Transfer Protocol (HTTP). However, gzip compression is supported by this service. Use of gzip compression may markedly speed up acquisition of data, particularly on large queries. It also is a more efficient use of USGS servers, so we appreciate when you are thoughtful enough to use it. Whether you can receive the data in gzip compressed formats depends on the capabilities of your client. Web browsers support gzip compression natively, but most regular users will use specialized utility programs like wget and curl to acquire data. If you can handle gzip compression, please place the following string into your HTTP request headers: Accept-Encoding: gzip, compress

curl and wget are typically used to download data. They may be configured to use gzip compression if the server will accept it. You may also explicitly have to tell it to use gzip compression. If so these tips should work:

• curl: try adding the argument: -H 'Accept-Encoding: gzip,deflate'
• wget: try adding the argument: --header="Accept-Encoding: gzip"

gzip files are typically returned as a file with a .gz file suffix unless you instruct your program to uncompress it. See the gzip man page for instructions on uncompressing .gz files.

Output

Please note that most recent data are marked provisional, so these data should be interpreted with caution as it is possible (although unlikely) to be incorrect. See the USGS Provisional Data Disclaimer page for more information.

When using format=waterml (the default format), data are returned in XML using the WaterML 1.1 schema . WaterML is a schema that has recently been adopted by the Open Geospatial Consortium . The crucial data are the instantaneous values, which can be found inside the <value> tag, which is nested inside the <timeSeries> tag, such as in this example:

<ns1:values>
  <ns1:value qualifiers="P" dateTime="2011-05-02">14.8</ns1:value>
  <ns1:qualifier qualifierID="0" ns1:network="NWIS" ns1:vocabulary="uv_rmk_cd">
    <ns1:qualifierCode>P</ns1:qualifierCode>
    <ns1:qualifierDescription>Provisional data subject to revision.</ns1:qualifierDescription>
  </ns1:qualifier>
  <ns1:method methodID="0">
    <ns1:methodDescription>sensor:null:7</ns1:methodDescription>
  </ns1:method>
</ns1:values>

By itself this imparts no information as to what the daily value represents, other than the statistic is provisional because qualifiers=“P”. We do know the value is 14.8 and it represents a daily value for May 2, 2011. We also need to know what was measured. The key to figuring this out is to examine the outer <timeSeries> tag, which contains some important information, shown below in bold:

<ns1:timeSeries name="USGS:01646500:00010:00002">
...
</ns1:timeSeries>

The name attribute contains a sequence of useful information with key fields delimited by colons. The pattern is <agencyCd>:<siteNo>:<parameterCd>:<statisticsCd>.

So this node contains data about site number 01646500 (Little Falls Pumping Station on the Potomac River) monitored by the USGS. Specifically it has a calculated daily statistic for USGS parameter 00010, which is water temperature in degrees Celsius. How do we know this? It is made clear inside the <variable> node within the <timeSeries> node.

<ns1:variable ns1:oid="45807042">
  <ns1:variableCode network="NWIS" vocabulary="NWIS:UnitValues" default="true" variableID="45807042">00010</ns1:variableCode>
  <ns1:variableName>Temperature, water, &#176;C</ns1:variableName>
  <ns1:variableDescription>Temperature, water, degrees Celsius</ns1:variableDescription>
  <ns1:valueType>Derived Value</ns1:valueType>
  <ns1:unit>
    <ns1:unitAbbreviation>deg C</ns1:unitAbbreviation>
  </ns1:unit>
  <ns1:options>
    <ns1:option name="Statistic" optionCode="00002">Minimum</ns1:option>
  </ns1:options><ns1:noDataValue>-999999.0</ns1:noDataValue>
</ns1:variable>

Since a daily value is a computation of many regular timeseries measurements, the next question is what statistic is being measured? A mean temperature? Maximum temperature? Minimum temperature? The statistics code is 00002, which the <options> tag nested inside the <variable> tag tells us is minimum (see above).

Putting it altogether, this means that for this site, the provisional minimum water temperature on May 2, 2011 was 14.8 degrees Celsius, or about 59 degrees Fahrenheit.

With other output formats, the location of the data will depend on the syntax of the format. You will need to inspect the format to locate the relevant data.

Error codes

Since this system uses Hypertext Transfer Protocol (HTTP), any application errors are reported in the HTTP headers. This means that when writing applications, it is important to first examine the HTTP status code that is returned in the HTTP response. The application server will return the error code along with a message describing the error in the event there is a problem. Programmers should always check the HTTP response code and if not a 200 handle the response as an exception. Among the status codes you may see:

Using the Web Service with Adobe Flex or the Flex API

Adobe Flex requires our server have a crossdomain.xml file indicating those domains that can access our web service using Adobe Flex. We are adding these on a case by case basis. If you need to access the service using Adobe Flex or the Flex API, please contact us using this form and indicate the domain of the server that will access the service.

CORS Support

This service supports the Cross-Origin Resource Sharing (CORS) specification. CORS permits browser-based asynchronous access to the service even though content originates from a server different than the one serving the web page. Otherwise the browser’s security controls would not allow content to come from USGS servers. Most, but not all browsers, support CORS. Some frameworks, like jQuery through the Ajax crossDomain setting, support CORS automatically.

URL Format

The URL must always be in this format:

https://waterservices.usgs.gov/nwis/dv/?<arguments>

where <arguments> are one or more HTTP GET parameter names and values based on the information below.

Specifying the URL Arguments

You specify the arguments that go in <arguments>.

• Each URL argument name is followed by an equal sign followed by one or more values for that argument. Where multiple values are allowed for the same argument, you can separate values with commas.
• URL arguments are separated by ampersands (&)
• The order of the URL arguments does not matter
• If a URL argument name does not match one of the names below, a HTTP 400 error code is returned

Here is an example of a valid URL that should return data:

https://waterservices.usgs.gov/nwis/dv/?site=01646500

URL argument names and argument values can be in upper, lower or mixed case. They will all be handled correctly. All of the following will yield the same result:

• https://waterservices.usgs.gov/nwis/dv/?stateCd=ny
• https://waterservices.usgs.gov/nwis/dv/?statecd=ny
• https://waterservices.usgs.gov/nwis/dv/?STATECD=ny
• https://waterservices.usgs.gov/nwis/dv/?stateCd=NY
• https://waterservices.usgs.gov/nwis/dv/?STATECD=NY
• https://waterservices.usgs.gov/nwis/dv/?stateCd=Ny

URL argument conventions

The following conventions are used below to document URL argument values:

arg1=[ a {,x | ,y} | b | c,d,...]

• square brackets [] are used to show a set of possible choices, with the pipe symbol | delineating exclusive choices. You must select one and only one of these choices.
• curved brackets {} are used to show optional elements. They also may be delineated with | indicating exclusive choices. If used, you may select one and only one of these choices.
• ... indicates more than item may be specified if items are delineated by commas. Note the limitation on the maximum number of argument values allowed below.

In the above example, these would be the allowed legal values:

• arg1=a
• arg1=a,x
• arg1=a,y
• arg1=b
• arg1=c
• arg1=c,d,e,f
• arg1=e,f

Major Filters

Single Site Queries

Want to only query one site? Use site (or sites) as your major filter, and put only one site number in the list! Example:

https://waterservices.usgs.gov/nwis/dv/?site=01646500

Multiple Site Queries

• Every multiple site query requires a major filter. Pick the major filter (sites, stateCd, huc, bBox or counties) that best retrieves the data for the sites that you are interested in.
• Further refine the query with minor filters, if needed.

Specifying Date Ranges

Many sites have periods of record that can be measured in decades, sometimes fifty years or more. Some sites have been measuring common parameters like streamflow continuously, others are seasonal in nature, and others have had periods when no funding was available to maintain the site so no data are available. Since daily values are by definition daily calculations, it makes no sense to request time periods in less than day increments. Please follow these rules for expressing dates with this service:

Format (format)

indent

Minor Filters

Additional filters can be applied after specifying a major filter. This further reduces the set of expected results. Users are encouraged to use minor filters because it allows more efficient use of this service.

Parameter Code (parameterCd)

Statistics Code (statCd)

Site Status (siteStatus)

Site Type (siteType)

Site was modified since (modifiedSince)

Agency Code (agencyCd)

Altitude (altMin and altMax)

Surface water filters

Drainage Area (drainAreaMin and drainAreaMax)

Groundwater Filters

Aquifer Code (aquiferCd)

Local Aquifer Code (localAquiferCd)

Well Depth (wellDepthMin and wellDepthMax)

Hole Depth (holeDepthMin and holeDepthMax)

Feedback

Please provide any feedback you have on this service using this form .

3 - Site Service

3.1 - Site Service Frequently Asked Questions

How do I retrieve just real-time sites?

Real-time sites are USGS hydrologic sites that regularly transmit automated measurements and observations. Currently there is no perfect mechanism for retrieving real-time sites. In particular &hasDataTypeCd=rt will not return only real-time sites, since rt is an alias for instantaneous values (iv). The NWIS Mapper uses an approach that shows “active” sites. However, active sites have different meanings depending on whether they serve automated (time-series) data or are regularly visited by humans for collecting data manually (discrete data). For real-time sites, users are usually interested in automated record stations. Thus combining this with a list of “active” sites that have been operational over a recent time period helps flag those sites that recorded traffic recently. This is close to being a list of current real-time sites. For example, to retrieve all active sites for Rhode Island in a Google Maps format that have recorded observations in the last week, use: https://waterservices.usgs.gov/nwis/site/?format=mapper&stateCd=ri&period=P1W&outputDataTypeCd=iv .

Is this service available as a Web Feature service?

Not in the format, but The USGS will be rolling out a number of new services to provide monitoring location metadata in the coming months and years.

How do I integrate your site service with commercial mapping services, like Google Maps?

For other mapping services, consult their application programming interfaces (APIs). Most mapping applications can map geographic data in a Keyhole Markup Language (KML) format. Using &format=ge for Google Earth and &format=gm for Google Maps will probably work with Yahoo, Microsoft and other public mapping services because they are KML-friendly.

What is a period of record?*

This is the time period that the site was actively maintained and used. Ideally, it is a period of contiguous time to the present, but sites may be discontinued for a while due to lack of funding or are seasonally affected by the weather. Note that sites where discrete (manual) measurements were made are not “active” in the traditional sense, unless automated equipment is collocated at the site. However they do represent dates when data was collected at the site. A site’s period of record may be further refined to show periods of record for different data types, for example a time range for collecting daily values and discrete water quality measurements.

How do I use the service to determine a list of sites that were removed?

Data collected at a site is always maintained as a matter of record. Sometimes a site is removed from public display. Unfortunately, there is no way to get a list of these sites from the service other than to keep your own copy and compare changes. Note that the modifiedSince argument effectively only shows new sites and sites where site information has changed.

What new features are planned for the site service?

Aside from the web feature service, which is likely to be a separate service, the following improvements are anticipated: Excel 2007 output, search by site name, polygons of enclosed sites, and minimum bounding rectangles. These features are expected in 2014 or 2015.

3.2 - Site Service Details

Site Web Service

You can use this service to retrieve information about the millions of hydrologic sites with data served by the USGS. This service provides USGS water data in the legacy RDB (tab-delimited) format that are currently available from the USGS Water Data for the Nation site External Link, as well as in a Google Maps and Google Earth friendly formats. More media types will follow after technical issues are resolved.

How the service works

• This is a REST-friendly service, which means it is URL accessible and can be run from a browser
• The service can return information about one or more sites in one request
• Data for historical, as well as currently active sites are available, including period of record information that describes the kinds of data collected at the site and date ranges when these data were collected.

With hundreds of thousands of sites across the nation, the amount of data available is very large. No one user is allowed to download all of the data with a single call. The service has consequently been designed and engineered to facilitate common mass queries, defaulting to returning a narrower set of data. You are encouraged to make your queries efficient too, mindful that many others need access to the data at the same time. Always specify the minimum amount of data you need in your request, using built in filters and date ranges to the maximum extent possible.

Testing the service

Probably the best way to learn how to use the service is to try it out!

The USGS provides this tool that allows you to generate syntactically correct calls to the service. You can even run the query live in your browser. The XML output may look somewhat strange if you are new to XML. When you have perfected your query you can copy and paste the URL into your application to get the precise data you need.

Test the service now

Enabling gzip compression

Typically data is downloaded as plain text via Hypertext Transfer Protocol (HTTP). However, gzip compression is supported by this service. Use of gzip compression may markedly speed up acquisition of data, particularly on large queries. It also is a more efficient use of USGS servers, so we appreciate when you are thoughtful enough to use it. Whether you can receive the data in gzip compressed formats depends on the capabilities of your client. Web browsers support gzip compression natively, but most regular users will use specialized utility programs like wget and curl to acquire data. If you can handle gzip compression, please place the following string into your HTTP request headers: Accept-Encoding: gzip, compress

curl and wget are typically used to download data. They may be configured to use gzip compression if the server will accept it. You may also explicitly have to tell it to use gzip compression. If so these tips should work:

• curl: try adding the argument: -H 'Accept-Encoding: gzip,deflate'
• wget: try adding the argument: --header="Accept-Encoding: gzip"

gzip files are typically returned as a file with a .gz file suffix unless you instruct your program to uncompress it. See the gzip man page for instructions on uncompressing .gz files.

Output

A web feature service is planned.

The most “data rich” format that is currently supported is the legacy USGS RDB (tab-delimited) format. The RDB format is documented here , and consists of a number of header comments that include column description information, followed by the relevant data. Fields are separated by invisible tab characters; records are separated with new line characters.

# US Geological Survey
#
# The Site File stores location and general information about ground water,
# surface water, and meteorological sites for sites in USA.
#
# File-format description: http://waterdata.usgs.gov/nwis/?tab_delimited_format_info
# Automated-retrieval info: http://waterdata.usgs.gov/nwis/?automated_retrieval_info
#
# Contact: gs-w_waterdata_support@usgs.gov
# retrieved: 2011-01-13 10:27:06 EST (nwisvatestws01)
#
# The following selected fields are included in this output:
# agency_cd -- Agency
# site_no -- Site identification number
# station_nm -- Site name
# site_tp_cd -- Site type
# dec_lat_va -- Decimal latitude
# dec_long_va -- Decimal longitude
# coord_acy_cd -- Latitude-longitude accuracy
# dec_coord_datum_cd -- Decimal Latitude-longitude datum
# alt_va -- Altitude of Gage/land surface
# alt_acy_va -- Altitude accuracy
# alt_datum_cd -- Altitude datum
#
agency_cd site_no station_nm site_tp_cd dec_lat_va dec_long_va coord_acy_cd dec_coord_datum_cd alt_va alt_acy_va alt_datum_cd 5s 15s 50s 7s 16s 16s 1s 10s 8s 3s 10s
USGS 01646500 POTOMAC RIVER NEAR WASH, DC LITTLE FALLS PUMP STA ST 38.94977778 -77.12763889 S NAD83 37.95 .01 NGVD29

Figure 1. Example of RDB output

The service also outputs in formats compatible with Google Earth and Google Maps by returning data in Keyhole Markup Language (KML) optimized for these mapping services. This allows you to show USGS sites on these and similar mapping products.

The Mapper format is a legacy XML format used by the USGS to support its National Water Information System Mapper product. However, these output formats contain minimal information about a site.

Many output formats from this service are straightforward and easy to understand. Google Map and Google Earth formats, for example, display minimal information for a site because they are intended to be used to show sites on a map. The RDB format can offer voluminous information that can be hard to understand and interpret.

Understanding site-level data

To interpret the RDB formats, you have to understand the meaning of the column headers in the output. Understanding basic site data is generally straightforward. In RDB, the column headers are described in comment lines that begin with a #.

• The first non comment line is a header line, followed by a line that describes the kind of data and the maximum size of data in that field (5s for a 5-character string, for example)
• The line after the header line are data lines. station_nm for example is POTOMAC RIVER NEAR WASH, DC LITTLE FALLS PUMP STA ST. This can be inferred because it is the third tab-delimited column in the header line, and the first data line contains this text in the third tab-position. If you don’t understand what station_nm is, the comments near the top of the file explain it should be interpreted as Site Name.
• In some outputs you will see repetition of comment and header lines after various data lines. Each of these blocks describes data immediately below it.

Understanding period of record output

Period of record information can be especially confusing. Period of record information describes what data was collected at a site for what time periods. Consider this partial output for this query . The data is tab-delimited and will line up by column if imported into a spreadsheet, but it hard to show on a web page. Items below have been bolded to show associations between headers and their data:

# end_date -- End date
# count_nu -- Record count
#
agency_cd site_no station_nm site_tp_cd dec_lat_va dec_long_va coord_acy_cd dec_coord_datum_cd alt_va alt_acy_va alt_datum_cd data_type_cd parm_cd stat_cd loc_nm medium_grp_cd parm_grp_cd srs_id access_cd begin_date end_date count_nu
5s 15s 50s 7s 16s 16s 1s 10s 8s 3s 10s 2s 5s 5s 30s 3s 3s 5n 4n 20d 20d 5n
USGS 01646500 POTOMAC RIVER NEAR WASH, DC LITTLE FALLS PUMP STA ST 38.94977778 -77.12763889 S NAD83 37.95 .01 NGVD29 dv 00010 00003 wat 1645597 0 1988-10-01 2011-04-08 7856

Figure 2. Period of Record output

There is a story here. Looked at the bolded text in the example:

• data_type_cd is associated with a value of “dv”. “dv” stands for daily values. So this site collects or collected daily values.
• parm_cd is the USGS parameter code defined here . 00010 corresponds to water temperature in degrees Celsius. So now we know this site this site collects daily values for water temperature.
• stat_cd stands for statistics code defined here . 00003 corresponds to mean, so now we know this site collects daily values for water temperature published as a mean (average) value for the day.
• begin_date means the date that this information started to be collected, which was October 1, 1988.
• end_date means end date and is April 8, 2011. If the site is active, this date will usually be the last full day. In short, mean water temperature is still being collected, so there is a continuous record for this kind of data since October 1, 1988 through April 8, 2011.
• count_nu is count number, or the number of mean daily values that are available for water temperature for this site, which is 7856, presumably one for each day from October 1, 1988 through April 8, 2011.

Starting late February 2024, the Site Service will stop retrieving new water quality samples/results data or updates to existing data for queries using hasDataTypeCd=qw.

Based on this information, an application might want to programatically start another application to fetch the daily value data using the Daily Values web service.

When GML is available as an output format, data will be encapsulated in XML tags but have similar names, so similar logic will apply.

Where to find USGS code information

• Most codes used by the service are documented on the USGS Water Data for the Nation Help site on the codes page .
• USGS Parameters (parm_cd) can be found here
• Certain definitions like site number and lat/long can be found here

Error codes

Since this system uses Hypertext Transfer Protocol (HTTP), any application errors are reported in the HTTP headers. This means that when writing applications, it is important to first examine the HTTP status code that is returned in the HTTP response. The application server will return the error code along with a message describing the error in the event there is a problem. Programmers should always check the HTTP response code and if not a 200 handle the response as an exception. Among the status codes you may see:

Using the Web Service with Adobe Flex or the Flex API

Adobe Flex requires our server have a crossdomain.xml file indicating those domains that can access our web service using Adobe Flex. We are adding these on a case by case basis. If you need to access the service using Adobe Flex or the Flex API, please contact us using this form and indicate the domain of the server that will access the service.

CORS Support

This service supports the Cross-Origin Resource Sharing (CORS) specification. CORS permits browser-based asynchronous access to the service even though content originates from a server different than the one serving the web page. Otherwise the browser’s security controls would not allow content to come from USGS servers. Most, but not all browsers, support CORS. Some frameworks, like jQuery through the Ajax crossDomain setting, support CORS automatically.

URL Format

The URL must always be in this format:

https://waterservices.usgs.gov/nwis/site/?<arguments>

where <arguments> are one or more HTTP GET parameter names and values based on the information below.

Specifying the URL Arguments

You specify the arguments that go in <arguments>.

• Each URL argument name is followed by an equal sign followed by one or more values for that argument. Where multiple values are allowed for the same argument, you can separate values with commas.
• URL arguments are separated by ampersands (&)
• The order of the URL arguments does not matter
• If a URL argument name does not match one of the names below, a HTTP 400 error is triggered.

Here is an example of a valid URL that should return data:

https://waterservices.usgs.gov/nwis/site/?site=01646500

URL argument names and URL argument values can be in upper, lower or mixed case. They will all be handled correctly. All of the following will yield the same result:

• https://waterservices.usgs.gov/nwis/site/?stateCd=ny
• https://waterservices.usgs.gov/nwis/site/?statecd=ny
• https://waterservices.usgs.gov/nwis/site/?STATECD=ny
• https://waterservices.usgs.gov/nwis/site/?stateCd=NY
• https://waterservices.usgs.gov/nwis/site/?STATECD=NY
• https://waterservices.usgs.gov/nwis/site/?stateCd=Ny

URL argument conventions

The following conventions are used below to document URL argument values:

arg1=[ a {,x | ,y} | b | c,d,...]

• square brackets [] are used to show a set of possible choices, with the pipe symbol | delineating exclusive choices. You must select one and only one of these choices.
• curved brackets {} are used to show optional elements. They also may be delineated with | indicating exclusive choices. If used, you may select one and only one of these choices.
• ... indicates more than item may be specified if items are delineated by commas. Note the limitation on the maximum number of argument values allowed below.

In the above example, these would be the allowed legal values:

• arg1=a
• arg1=a,x
• arg1=a,y
• arg1=b
• arg1=c
• arg1=c,d,e,f
• arg1=e,f

Major Filters

Single Site Queries

Want to only query one site? Use site (or sites) as your major filter, and put only one site number in the list! Example:

https://waterservices.usgs.gov/nwis/site/?site=01646500

Multiple Site Queries

• Every multiple site query requires a major filter. Pick the major filter (sites, stateCd, huc, bBox or counties) that best retrieves the data for the sites that you are interested in.
• You can have more than one major filter per query. However, you cannot replicate a particular major filter in the query. The effect is an AND, reducing the results returned. Ex: &stateCd=nh&huc=01060003 returns sites in New Hampshire for Piscataqua-Salmon Falls.
• Further refine the query with minor filters, if needed.

Outputs

Format (format)

Show expanded Site Output (siteOutput)

Show Period of Record (seriesCatalogOutput)

Show one or more data types (outputDataTypeCd)

Minor Filters

Minor filters further reduce the set of expected results and are applied after data is first filtered by the major filter you specified. You are encouraged to use minor filters to minimize the amount of data sent.

Date Filters

Start Date (startDt)

End Date (endDt)

Period (period)

Site was modified since (modifiedSince)

Site Name (siteName)

Site Name Match Operator (siteNameMatchOperator)

Site Status (siteStatus)

Site Type (siteType)

Has one or more data types (hasDataTypeCd)

Parameter Code (parameterCd)

Agency Code (agencyCd)

Altitude (altMin and altMax)

Surface water filters

Drainage Area (drainAreaMin and drainAreaMax)

Groundwater Filters

Aquifer Code (aquiferCd)

Local Aquifer Code (localAquiferCd)

Well Depth (wellDepthMin and wellDepthMax)

Hole Depth (holeDepthMin and holeDepthMax)

Feedback

Please provide any feedback you have on this service using this form .

4 - Statistics

4.1 - Statistics Service Frequently Asked Questions

Why is only tab-delimited (RDB) format available?

The USGS is starting with a commonly used internal format for serving statistical data for this service. Schemas need to be created to represent statistical water data in XML and JSON. When these are completed they will be integrated into the service.

Why are there no statistics served for discrete sites?

Discrete sites are those where human (manual) measurements are made. However, to generate a statistic it must be based on looking at many values over a time period. Automated time-series sites perform this function, making statistics like minimum, maximum and mean easy to calculate. Months or years may go by before a human visits the site to take a manual measurement. Consequently, a site must serve daily values for statistics to be generated.

Why are provisional data not included in statistics?

Provisional data, as the name implies, is subject to change. USGS hydrologists frequently change provisional data, for example to remove aberrant spikes or to estimate flow when a site is ice affected. After these changes, the data are marked as approved and statistics can be calculated from these values. Since some values will be incorrect or missing if marked provisional, statistics based on provisional data will be misleading, so they are not allowed.

Why are there no statistics for a period because only a few daily values are missing?

To be faithful to the intent of a statistic, it should be calculated based on a complete set of values. For example, a monthly statistics for a given month and year should be based on accurate daily values for all days in the month for the year. Consequently, if there is not a complete record of daily values for that month, statistics for that month are not served. However, the service allows this to be overridden by supplying the missingData argument.

How are statistics calculated? Are they rounded? What rules apply?

See this topic in the USGS Water Data for the Nation help system.

4.2 - Statistics Service Details

Statistics Web Service

You can use this service to retrieve daily, monthly or annual statistics for a USGS site and parameter.

To provide statistics, the site must serve time-series data, i.e. data is collected via an automated means and at regular intervals (typically every 15 minutes). You can use the daily values test tool or the site service test tool to find these sites. When using the site service test tool, make sure your query is qualifed with the argument &hasDataTypeCd=dv.

The service serves statistics based on approved data only. Recent data is considered provisional and is thus not approved. In general, you may find periods of unapproved statistics when requesting statistics within a year or less from today. Unapproved statistics will not show. The value may be left blank depending on the service called.

How the service works

• This is a REST-friendly service, which means it is URL accessible and can be run from a browser
• The service can return statistics for one or more sites in one request
• Data for historical, as well as currently active sites are available, providing the site collected time-series data.

With hundreds of thousands of qualifying sites across the nation, the amount of data available is very large. No one user is allowed to download all of the data with a single call. The service has consequently been engineered to facilitate common mass queries, defaulting to returning a narrower set of data. You are encouraged to make your queries efficient too, mindful that many others need access to the data at the same time. Always specify the minimum amount of data you need in your request, using built in filters and date ranges to the maximum extent possible.

Testing the service

Probably the best way to learn how to use the service is to try it out!

The USGS provides this tool that allows you to generate syntactically correct calls to the service. You can even run the query live in your browser. The XML output may look somewhat strange if you are new to XML. When you have perfected your query you can copy and paste the URL into your application to get the precise data you need.

Test the service now

Enabling gzip compression

Typically data is downloaded as plain text via Hypertext Transfer Protocol (HTTP). However, gzip compression is supported by this service. Use of gzip compression may markedly speed up acquisition of data, particularly on large queries. It also is a more efficient use of USGS servers, so we appreciate when you are thoughtful enough to use it. Whether you can receive the data in gzip compressed formats depends on the capabilities of your client. Web browsers support gzip compression natively, but most regular users will use specialized utility programs like wget and curl to acquire data. If you can handle gzip compression, please place the following string into your HTTP request headers: Accept-Encoding: gzip, compress

curl and wget are typically used to download data. They may be configured to use gzip compression if the server will accept it. You may also explicitly have to tell it to use gzip compression. If so these tips should work:

• curl: try adding the argument: -H 'Accept-Encoding: gzip,deflate'
• wget: try adding the argument: --header="Accept-Encoding: gzip"

gzip files are typically returned as a file with a .gz file suffix unless you instruct your program to uncompress it. See the gzip man page for instructions on uncompressing .gz files.

Daily, monthly and annual statistics

Daily statistics are statistics based on the day of the year. For example, if you request a daily mean statistic for streamflow for January 1st for a site, the mean streamflow for January 1st for each year of interest is acquired and the mean of all these values is then calculated. In this case the daily statistics is the mean of all the means for that date. If you asked for minimum, you would get the minimum of all the minimums for the date, etc.

Monthly statistics are calculations based on a month and year. It answers questions like “For February 1999 what was the mean streamflow?” Using mean as an example, you will get a mean (average) of all the daily means for the site for February 1999.

Annual statistics are based on a calendar year or water year. For a given year you will get back a single statistic for a site and a parameter, such as the mean streamflow for a site for calendar year 2010.

Sometimes gages may be down or inaccessible due to equipment malfunctions or other issues. You have the option to calculate monthly or annual statistics if there is missing data using the missingData argument. Otherwise there must be a complete set of approved statistics for the type of statistic wanted to serve that statistic. If it cannot be served, the value is generally left blank or null.

Interpreting the output

Currently only tab-delimited data can be downloaded using this service. XML and JSON outputs will be added in a subsequent release. Regardless of the format downloaded, only the way it is encoded and marked up differs.

Daily statistics

Caution: with daily statistics, the years in the output are in water years, not calendar years. Water years begin October 1 of the year before the calendar year and end September 30 of the calendar year. As a consequence, if concerned about the actual calendar year for the statistic, you must subtract 1 from the year shown if the date is between October 1 and December 31.

This query returns default statistics for site 01646500, at the Little Falls Pumping Station on the Potomac River, specifically streamflow or discharge. The site number is identified in Column B and the parameter being measured (streamflow) is identified in column C (00060). Row 1 contains column names and Row 2 contains some information about the type of data in the column. For example, 5s means expect a string (set of characters) not to exceed 5 characters. Data begin with Row 3.

Column F indicates the month of the statistic and column G indicates the day in the month of the statistic. So Row 3 contains statistics for January 1st across the entire period of record, which in this case was from the beginning of water year 1931 (October 1, 1930) to end of water year 2013 (September 30, 2013), as indicated by Columns H and I. The last row, Row 21, contains statistics for December 31. Many rows were removed for readability.

Mean is the same as average. So Cell O3 indicates that the average streamflow for this site on January 1st is 13,400 cubic feet per second across more than eighty years of record.

What was the maximum flow on January 1st? Cell L3 tells us: 118,000 cubic feet per second and this record occurred in 1943 (Cell K3). Similarly, minimum flow for January 1st occurred in 1966 (Cell M3) when streamflow was 940 cubic feet per second (Cell N3).

Columns P through X indicate percentiles, indicating the value below which a given percentage of observations in a group of observations fall. For example, the 20th percentile is the value below which 20 percent of the observations may be found, which for January 1st is 3,490 cubic feet per second (Cell R3). The median value is the same as the 50th percentile. The median value for January 1st is 9,050 cubic feet per second (Cell T3). The percentiles shown are provided by default. There is no year associated with percentile, just as there is no year associated with the mean, as it makes sense only for minimum or maximum values.

The “loc_web_ds” column (Column D) is normally blank. Some sites may measure from different locations, such as the right bank and the left bank of a river. If this site has multiple measurement locations, you will see text here describing the site location. The “dd_nu” column (Column E) indicates the Data Descriptor associated with streamflow at this site. Generally, this is of interest only to USGS scientists and can be ignored.

What is “count_nu”? In this case it represents the 83 years of statistics, or its period of record in years. This will stay the same except in the peculiar case of February 29. Since this is recorded only every four years, the number is smaller, 21 in this case.

Daily statistics are rounded using these rounding rules .

Monthly statistics

Annual statistics

Error codes

Since this system uses Hypertext Transfer Protocol (HTTP), any application errors are reported in the HTTP headers. This means that when writing applications, it is important to first examine the HTTP status code that is returned in the HTTP response. The application server will return the error code along with a message describing the error in the event there is a problem. Programmers should always check the HTTP response code and if not a 200 handle the response as an exception. Among the status codes you may see:

Using the Web Service with Adobe Flex or the Flex API

Adobe Flex requires our server have a crossdomain.xml file indicating those domains that can access our web service using Adobe Flex. We are adding these on a case by case basis. If you need to access the service using Adobe Flex or the Flex API, please contact us using this form and indicate the domain of the server that will access the service.

CORS Support

This service supports the Cross-Origin Resource Sharing (CORS) specification. CORS permits browser-based asynchronous access to the service even though content originates from a server different than the one serving the web page. Otherwise the browser’s security controls would not allow content to come from USGS servers. Most, but not all browsers, support CORS. Some frameworks, like jQuery through the Ajax crossDomain setting, support CORS automatically.

URL Format

The URL must always be in this format:

https://waterservices.usgs.gov/nwis/stats/?<arguments>

where <arguments> are one or more HTTP GET parameter names and values based on the information below.

Specifying the URL Arguments

You specify the arguments that go in <arguments>.

• Each URL argument name is followed by an equal sign followed by one or more values for that argument. Where multiple values are allowed for the same argument, you can separate values with commas.
• URL arguments are separated by ampersands (&)
• The order of the URL arguments does not matter
• If a URL argument name does not match one of the names below, a HTTP 400 error code is returned

Here is an example of a valid URL that should return data:

[https://waterservices.usgs.gov/nwis/stat/?site=01646500

URL argument names and argument values can be in upper, lower or mixed case. They will all be handled correctly. All of the following will yield the same result:

• https://waterservices.usgs.gov/nwis/stat/?sites=07144100&parameterCd=00060
• https://waterservices.usgs.gov/nwis/stat/?sites=07144100&parametercd=00060
• https://waterservices.usgs.gov/nwis/stat/?sites=07144100&PARAMETERCD=00060

URL argument conventions

The following conventions are used below to document URL argument values:

arg1=[ a {,x | ,y} | b | c,d,...]

• square brackets [] are used to show a set of possible choices, with the pipe symbol | delineating exclusive choices. You must select one and only one of these choices.
• curved brackets {} are used to show optional elements. They also may be delineated with | indicating exclusive choices. If used, you may select one and only one of these choices.
• ... indicates more than item may be specified if items are delineated by commas. Note the limitation on the maximum number of argument values allowed below.

In the above example, these would be the allowed legal values:

• arg1=a
• arg1=a,x
• arg1=a,y
• arg1=b
• arg1=c
• arg1=c,d,e,f
• arg1=e,f

Major Filters

Single Site Queries

Want to only query one site? Use site (or sites) as your major filter, and put only one site number in the list! Example:

https://waterservices.usgs.gov/nwis/iv/?site=01646500

Multiple Site Queries

Unlike the other services on this site, at this time the statistics service is limited to accepting a list of up to ten site numbers only. At least one site number must be supplied.

Specifying Date Ranges for calculating statistics

Specifying dates is optional

Choosing to specify date ranges is optional. If you don’t specify a date range then statistics will be calculated against the entire period of record for a site and the parameter(s) of interest. So one site may have fifty years of record and its statistics will be calculated based on that period, while another site may have ten years of record and its statistics will be calculated over its 10 years. If you want to constrain statistics to be calculated over a narrower period of interest than the period of record, specify dates using startDt and endDt arguments.

Annual statistics: calendar year and water year, what’s the difference?

Annual statistic can be calculated against a calendar year or water year. You can select the type of year you want with the statYearType parameter.

• Calendar years begin January 1 and end December 31. Calendar years are assumed by default.
• Water years are the same as federal fiscal years. Federal fiscal years begin October 1 of the previous year and end on September 30 of the current year. For example, Water Year 2013 began October 1, 2012 and ended September 30, 2013. Water years are selected by including the URL argument statYearType=water. Thus when you specify statReportType=annual&startDt=2005&endDt=2005&statYearType=water the annual statistics will be based on a year beginning October 1, 2004 and ending September 30, 2005.

Start Date (startDt)

End Date (endDt)

Rules for specifying dates

Dates are specified in an ISO-8601 date format. You can use the format YYYY for years, YYYY-MM for months and YYYY-MM-DD for days. Use the startDt and endDt URL arguments when you need to specify period of interest dates that are less than the period of record. startDt is the start date, month or year. endDt is the end date, month or year. endDt must always be equal to or greater than startDt.

Daily statistics:

• Use startDt and endDt to a day, month or year granularity. For example, startDt=2010-03-01. If you use less granularity, the start of the month or year will be assumed. startDt=2010-03 yields startDt=2010-03-01 and startDt=2010 yields startDt=2010-01-01.
• For endDt, if you use less granularity then the end of the month or year will be assumed. endDt=2010-11 yields endDt=2010-11-30 and endDt=2010 yields endDt=2010-12-31.

Monthly statistics:

• Do not specify a startDt or endDt to a day’s granularity. For example, do not do this: startDt=2010-03-01. You will get an error.
• Use startDt and endDt to a month granularity. For example, use startDt=2010-03.
• For startDt, if you use less granularity than month, then the start of the calendar year will be assumed. startDt=2010 yields endDt=2010-01.
• For endDt, if you use less granularity than month then the end of the calendar year will be assumed. endDt=2010 yields endDt=2010-12.

Annual statistics:

Annual statistics can be calculated against a calendar year or a water year. The statYearType argument is used to set the type of annual statistics desired.

• Specify only to the granularity of a year, ex: startDt=2010 or endDt=2012. Specifying these dates in a YYYY-MM or YYYY-MM-DD format will result in an error.
• For calendar year statistics (statYearType=calendar), which is the default, statistics are calculated based on years starting January 1 and ending December 31.
• For water year statistics (statYearType=water), statistics are calculated based on years starting October 1 of the previous year and ending September 30 of the current year. Thus, the meaning of the year specified in startDt and endDt will change, even though you will use the YYYY format to indicate the year. For example, startDt=2010&endDt=2013&statYearType=water means that annual statistics are calculated based on a calendar period from October 1, 2009 through September 30, 2013.

Defaults for start and end dates

Daily, monthly and annual statistics all depend on daily values, so daily values must exist and they must be approved. Moreover, they will only exist if a site has been recording statistics for the entire day. This helps explain the logic on defaults for dates with this service.

For start date (startDt):

• For daily statistics, the default is the first full day of the period of record. For example, if a site started recording data on April 13, 1995 at 12:00 local time then the default startDt will be startDt=1995-04-14
• For monthly statistics, the default start month is the first month after a site started recording data. For example, if a site started recording data on April 13, 1995 then the default startDt will be startDt=1995-05
• For annual statistics, the default start year is the first year after a site started recording data. For example, if a site started recording data on April 13, 1995 then the default startDt will be startDt=1996. Note that annual statistics can be calculated over calendar years or water years.

For end date (endDt):

• For daily statistics, the default is the last day of the full period of record. For example, if a site stopped recording data on September 12, 2007 at 12 PM local time, then the default endDt will be endDt=2007-09-11
• For monthly statistics, the default end month is the first month before a site stopped recording data. For example, if a site stopped recording data on September 12, 2007 then the default endDt will be endtDt=2007-08
• For annual statistics, the default end year is the first year before a site stopped recording data, unless it stopped at the end of the year exactly. For example, if a site stopped recording data on September 12, 2007 then the default endDt will be endDt=2006. Note that annual statistics can be calculated over calendar years or water years using the statYearType and startDt URL arguments.

Note that you can ask for annual and monthly statistics despite missing data within the month or year with the missingData argument. In some cases this will change default behavior if there are enough partial values to create a statistic for a start or end month.

Statistics and date ranges

Regardless of the statistics type wanted (daily, monthly or annual), statistics are computed against the period of interest by using startDt and endDt arguments. If not specified, the start and end dates for the period of record are used.

You can specify a range shorter or longer than a site’s period of record using the startDt and endDt arguments.

• If the period of interest is less than the period of record, it will calculate statistics across that date range, for example startDt=1960&endDt=1979 will calculate over those twenty years, even if the full period of record is from 1950 to 2009.
• If the period of interest is greater than a site’s actual period of record, the actual period of record will be used. For example, if the period of record is from 1995-2004 and you specify &startDt=1990&endDt=2009, values will be computed against the ten years of record from 1995 to 2004. After all, you can only generate statistics based on data actually available.

Date examples

• ?site=01646500 // Returns daily statistics for all parameters for the entire period of record of the site. Daily statistics are the default statistics type.
• ?site=01646500&statReportType=annual // Returns calendar year annual statistics for all parameters for the entire period of record
• ?site=01646500&statReportType=annual&startDt=2010 // Returns annual statistics for all parameters starting with calendar year 2010 through the last full calendar year of approved statistics
• ?site=01646500&statReportType=annual&statYearType=water // Returns annual statistics for all parameters by water year, where each water year actually begins October 1 of the previous calendar year and ends September 30 of the calendar year
• ?site=01646500&statReportType=annual&startDt=2010&endDt=2010 // Returns annual statistics for all parameters for calendar year 2010 only
• ?site=01646500&statReportType=monthly&startDt=2010-01 // Returns monthly statistics for all parameters based on the calendar year 2010 through the last full month of approved statistics
• ?site=01646500&statReportType=monthly&startDt=2010&endDt=2010 // Returns monthly statistics for all parameters based on calendar year 2010 only
• ?site=01646500&statReportType=monthly&startDt=2010-01&endDt=2010-12 // Returns monthly statistics for all parameters based on calendar year 2010 only
• ?site=01646500&statReportType=monthly&startDt=2010-05&endDt=2010-06 // Returns monthly statistics for all parameters for May and June 2010
• ?site=01646500&startDt=2010-01-01 // Returns daily statistics for all parameters for calendar year 2010 through to the present year
• ?site=01646500&startDt=2010-01-01&endDt=2010-12-31 // Returns daily statistics for all parameters for calendar year 2010 only. Note: since this is within a calendar year, the data returned should be the same as a daily values, so it would be better to use the daily values service.
• ?site=01646500&startDt=2010-01&endDt=2010-12 // Same as above
• ?site=01646500&startDt=2010&endDt=2010 // Same as above
• ?site=01646500&startDt=2010-05-15&endDt=2010-05-17 // Returns daily statistics for all parameters from May 15, 2010 through May 17, 2010. Note: since this is within a calendar year, the data returned should be the same as a daily value, so it would be better to use the daily values service.

Format (format)

Indenting (indent)

Statistical Controls

These controls influence which statistics are served and how statistics are calculated.

Statistics Report Type (statReportType)

Statistics Type (statType)

Missing Data (missingData)

Statistics Year Type (statYearType)

Minor Filters

Additional filters can be applied after specifying a major filter. This further reduces the set of expected results. Users are encouraged to use minor filters because it allows more efficient use of this service. Note: since only site numbers can be supplied as a major filter at this time, minor filters other than parameter code have limited usefulness.

Parameter Code (parameterCd)

Site Status (siteStatus)

Site Type (siteType)

Agency Code (agencyCd)

Altitude (altMin and altMax)

Surface water filters

Drainage Area (drainAreaMin and drainAreaMax)

Groundwater Filters

Aquifer Code (aquiferCd)

Local Aquifer Code (localAquiferCd)

Well Depth (wellDepthMin and wellDepthMax)

Hole Depth (holeDepthMin and holeDepthMax)

Feedback

Please provide any feedback you have on this service using this form .

5 - Groundwater Levels Service

5.1 - Ground Water Levels Frequently Asked Questions

Why does the service support WaterML 1.2 when there is no version 1.2 of WaterML?

WaterML is used to describe time-series water data. The groundwater levels service provides discrete (manual) measurements, which can be considered irregular instead of regular time series measurements. To accommodate the difference, a slight change was made to the WaterML 1.1 schema. This USGS revision is referenced by placing the schema in the USGS namespace only.

Where can I find automated groundwater measurements? These are available in the instantaneous values web service. Search on common groundwater parameters such as 72019 (depth of water below land surface). Currently these are limited to data served since October 1, 2007. Older groundwater data may be available through an inquiry with the local USGS water science center by writing gs-w-xx_nwisweb_data_inquiries@usgs.gov where xx is the state postal code (such as “ny”) of interest.

5.2 - Groundwater Levels Service Details

Groundwater Levels Service

You can use this service to retrieve historical manually-recorded groundwater levels from hydrologic sites served by the USGS. If you are looking to retrieve data for real-time or recent groundwater levels recorded with automated equipment on a regular basis, please use the instantaneous values web service .

This service provides USGS groundwater data in WaterML, the legacy RDB (tab-delimited) format, and in a Web 2.0 friendly JSON format (which renders WaterML as JSON). The data in the service is identical to data available from the USGS Water Data for the Nation site .

How the service works

• This is a REST-friendly service, which means it is URL accessible and can be run from a browser
• The service can return information about one or more sites in one request

With millions of groundwater levels collected at sites across the nation, the amount of data available is very large. No one user is allowed to download all of the data with a single call. The service has consequently been designed and engineered to facilitate common mass queries, defaulting to returning a narrower set of data. You are encouraged to make your queries efficient too, mindful that many others need access to the data at the same time. Always specify the minimum amount of data you need in your request, using built in filters and date ranges to the maximum extent possible.

Testing the service

Probably the best way to learn how to use the service is to try it out!

The USGS provides this tool that allows you to generate syntactically correct calls to the service. You can even run the query live in your browser. The XML output may look somewhat strange if you are new to XML. When you have perfected your query you can copy and paste the URL into your application to get the precise data you need.

Test the service now

Enabling gzip compression

Typically data is downloaded as plain text via Hypertext Transfer Protocol (HTTP). However, gzip compression is supported by this service. Use of gzip compression may markedly speed up acquisition of data, particularly on large queries. It also is a more efficient use of USGS servers, so we appreciate when you are thoughtful enough to use it. Whether you can receive the data in gzip compressed formats depends on the capabilities of your client. Web browsers support gzip compression natively, but most regular users will use specialized utility programs like wget and curl to acquire data. If you can handle gzip compression, please place the following string into your HTTP request headers: Accept-Encoding: gzip, compress

curl and wget are typically used to download data. They may be configured to use gzip compression if the server will accept it. You may also explicitly have to tell it to use gzip compression. If so these tips should work:

• curl: try adding the argument: -H 'Accept-Encoding: gzip,deflate'
• wget: try adding the argument: --header="Accept-Encoding: gzip"

gzip files are typically returned as a file with a .gz file suffix unless you instruct your program to uncompress it. See the gzip man page for instructions on uncompressing .gz files.

Output

WaterML

When using format=waterml (the default format), data are returned in XML using the WaterML 1.1 schema . WaterML is a schema that has recently been adopted by the Open Geospatial Consortium . The crucial data are the instantaneous values, which can be found inside the <value> tag, which is nested inside the <timeSeries> tag, such as in this example:

<ns1:values>
  <ns1:value dateTime="1968-08-28T00:00:00.000" sourceCode="D">145</ns1:value>
  <ns1:value dateTime="1980-09-16T00:00:00.000" sourceCode="D">311.5</ns1:value>
  <ns1:method methodID="1"/>
</ns1:values>

Please also note that the values found for this site are associated with the variable node that precedes it. The variable node indicates the parameter that is being measured or calculated. In this case, both the <ns1:values> and <ns1:variable> nodes are nested within the <ns1:timeSeries> node.

<ns1:variable ns1:oid="52331280">
  <ns1:variableCode network="NWIS" vocabulary="NWIS:UnitValues" default="true" variableID="52331280">72019</ns1:variableCode>
  <ns1:variableName>Depth to water level, ft below land surface</ns1:variableName>
  <ns1:variableDescription>Depth to water level, feet below land surface</ns1:variableDescription>
  <ns1:valueType>Derived Value</ns1:valueType>
  <ns1:unit>
    <ns1:unitCode>ft</ns1:unitCode>
  </ns1:unit>
  <ns1:options>
    <ns1:option name="Statistic" optionCode="00011"/>
  </ns1:options>
  <ns1:noDataValue>-999999.0</ns1:noDataValue>
</ns1:variable>

Putting it altogether, this means that for this site, two manual groundwater level measurements were recorded: 145 feet below the surface of the land on August 28, 1968 and 311.5 feet below the surface of the land on September 16, 1980.

The option code of 00011 indicates the type of statistics code that was measured. For groundwater levels, the statistics code of 00011 will typically be used, as it represents discrete manual measurements. A list of statistics codes can be found here .

With other output formats, the location of the data will depend on the syntax of the format. You will need to inspect the format to locate the relevant data.

Note: the current version of WaterML is 1.1. WaterML 1.2 is an unofficial variant created by the USGS needed to accommodate the discrete data provided by this service, and is referenced in a USGS name space.

Tab-delimited (RDB)

Returning tab delimited data provides the same data is a more succinct format. A partial excerpt is shown:

# ---------------------------------- WARNING ----------------------------------------
# The data you have obtained from this automated U.S. Geological Survey database
# have not received Director's approval and as such are provisional and subject to
# revision.  The data are released on the condition that neither the USGS nor the
# United States Government may be held liable for any damages resulting from its use.
# Additional info: http://waterdata.usgs.gov/nwis/help/?provisional
#
# File-format description: http://waterdata.usgs.gov/nwis/?tab_delimited_format_info
# Automated-retrieval info: http://waterdata.usgs.gov/nwis/?automated_retrieval_info
#
# Contact:   gs-w_waterdata_support@usgs.gov
# retrieved: 2012-02-27 14:05:09 EST (nwisvatestws01)
#
# US Geological Survey groundwater levels
#
# Data for the following 1 site(s) are contained in this file
#    USGS 375907091432201 Rolla Industrial Park
# -----------------------------------------------------------------------------------
#
# The fields in this file include:
# ---------------------------------
# agency_cd     Agency Code
# site_no       USGS site number
# site_tp_cd    Site Type Code
# lev_dt        Date level measured
# lev_tm        Time level measured
# lev_tz_cd     Time datum
# lev_va        Water level value in feet below land surface
# sl_lev_va     Water level value in feet above specific vertical datum
# sl_datum_cd   Referenced vertical datum
# lev_status_cd The status of the site at the time the water level was measured
# lev_agency_cd The agency code of the person measuring the water level
#
# Discrete groundwater status codes (lev_status_cd) included in this output:
#
#     ""         The reported water-level measurement represents a static level
#
# Agency codes (lev_agency_cd) included in this output:
#
#     MO005      MO DIV OF GEOL & LAND SURVEY, DEPT-NAT RESO, MO
#     USGS       US GEOLOGICAL SURVEY
#
# Referenced vertical datum codes (sl_datum_cd) included in this output:
#
#
#
# Site Type Codes (site_tp_cd) included in this output:
#
#     GW         Well
#
agency_cd site_no site_tp_cd lev_dt lev_tm lev_tz_cd lev_va sl_lev_va sl_datum_cd lev_status_cd lev_agency_cd
5s 15s 6s 10d 4d 5s 12s 12s 10s 1s 5s
USGS 375907091432201 GW 1968-08-28 145 MO005
USGS 375907091432201 GW 1980-09-16 311.5 USGS

Tab-delimited output does not indicate which USGS parameter code is associated with the measurements. This is true on the USGS Water Data for the Nation site as well. WaterML requires an associated <variable> tag with a measurement. To accommodate this request, the USGS assigns these data an appropriate USGS parameter code to the output, which is typically 72019.

Error codes

Since this system uses Hypertext Transfer Protocol (HTTP), any application errors are reported in the HTTP headers. This means that when writing applications, it is important to first examine the HTTP status code that is returned in the HTTP response. The application server will return the error code along with a message describing the error in the event there is a problem. Programmers should always check the HTTP response code and if not a 200 handle the response as an exception. Among the status codes you may see:

Using the Web Service with Adobe Flex or the Flex API

Adobe Flex requires our server have a crossdomain.xml file indicating those domains that can access our web service using Adobe Flex. We are adding these on a case by case basis. If you need to access the service using Adobe Flex or the Flex API, please contact us using this form and indicate the domain of the server that will access the service.

CORS Support

This service supports the Cross-Origin Resource Sharing (CORS) specification. CORS permits browser-based asynchronous access to the service even though content originates from a server different than the one serving the web page. Otherwise the browser’s security controls would not allow content to come from USGS servers. Most, but not all browsers, support CORS. Some frameworks, like jQuery through the Ajax crossDomain setting, support CORS automatically.

URL Format

The URL must always be in this format:

https://waterservices.usgs.gov/nwis/gwlevels?<arguments>

where <arguments> are one or more HTTP GET parameter names and values based on the information below. Specifying the URL Arguments

Specifying the URL Arguments

You specify the arguments that go in <arguments>.

• Each URL argument name is followed by an equal sign followed by one or more values for that argument. Where multiple values are allowed for the same argument, you can separate values with commas.
• URL arguments are separated by ampersands (&)
• The order of the URL arguments does not matter
• If a URL argument name does not match one of the names below, a HTTP 400 error is triggered.

Here is an example of a valid URL that should return data:

https://waterservices.usgs.gov/nwis/gwlevels?sites=375907091432201

URL argument names and URL argument values can be in upper, lower or mixed case. They will all be handled correctly. All of the following will yield the same result:

• https://waterservices.usgs.gov/nwis/gwlevels/?format=rdb&siteStatus=active&countyCd=51059
• https://waterservices.usgs.gov/nwis/gwlevels/?format=rdb&siteStatus=active&countycd=51059
• https://waterservices.usgs.gov/nwis/gwlevels/?format=rdb&siteStatus=active&COUNTYCD=51059

URL argument conventions

The following conventions are used below to document URL argument values:

arg1=[ a {,x | ,y} | b | c,d,...]

• square brackets [] are used to show a set of possible choices, with the pipe symbol | delineating exclusive choices. You must select one and only one of these choices.
• curved brackets {} are used to show optional elements. They also may be delineated with | indicating exclusive choices. If used, you may select one and only one of these choices.
• ... indicates more than item may be specified if items are delineated by commas. Note the limitation on the maximum number of argument values allowed below.

In the above example, these would be the allowed legal values:

• arg1=a
• arg1=a,x
• arg1=a,y
• arg1=b
• arg1=c
• arg1=c,d,e,f
• arg1=e,f

Major Filters

Single Site Queries

Want to only query one site? Use site (or sites) as your major filter, and put only one site number in the list! Example:

https://waterservices.usgs.gov/nwis/gwlevels/?site=01646500

Multiple Site Queries

• Every query requires a major filter. Pick the major filter (sites, stateCd, huc, bBox, counties) that best retrieves the data for the sites that you are interested in.
• You can have only one major filter per query. If you specify more than one major filter, you will get an error.

<td>1</td>
<td>20</td>
<td>&amp;countyCd=51059,51061 </td>

Outputs

format

Minor Filters

Minor filters further reduce the set of expected results and are applied after data is first filtered by the major filter you specified. You are encouraged to use minor filters to minimize the amount of data sent.

Specifying Date Ranges

Site Status (siteStatus)

Parameter Code (parameterCd)

Groundwater Site Type (siteType)

Agency Code (agencyCd)

Altitude (altMin and altMax)

Well Depth (wellDepthMin and wellDepthMax)

Hole Depth (holeDepthMin and holeDepthMax)

Aquifer Code (aquiferCd)

Local Aquifer Code (localAquiferCd)

Feedback

Please provide any feedback you have on this service using this form .

6 - General Questions

Are new services planned? If so what will they be?

A number of new web services are in active development. These services are being piloted on (https://labs.waterdata.usgs.gov )

How can I get notified of changes to this system? There are two ways:

1. Subscribe to the site’s news feed to learn when new features are released.
2. For general news, site outage information etc. news on USGS water services is also sent to email subscribers via the USGS Water Data Notification Service External Link. This is a better method if you want to know what will be coming up. New services may be open to public testing prior to deployment, in which case your testing may prove important to the quality and usability of new services.

How do I find out if I am being blocked?

You should see a HTTP 403 Access Forbidden error.

Why am I being blocked?

Blocking access to a service should only occur if the USGS believes that your use of the service is so excessive that it is seriously impacting others using the service.

How do I get unblocked?

To get unblocked, send us the URL you are using along with the IP using this form. We may require changes to your query and frequency of use in order to give you access to the service again.

Do you have guidelines for acceptable usage?

For small requests (e.g., up to 7 or 14 days for a single site) limit requests to a maximum of 5-10 per second at a steady rate. A maximum burst rate of to 40 or 50 request per second is OK, but not for extended periods of time.

My scripts time out. Is your service running slowly?

Possibly. It could also be that the client you are using is not configured to handle redirects. In this case, there are two approaches to handle redirect:

1. Fully qualify the URL by adding a / before the ? in the URL, ex: https://waterservices.usgs.gov/nwis/iv/?format=waterml,1.1&sites=01646500 rather than https://waterservices.usgs.gov/nwis/iv?format=waterml,1.1&sites=01646500 .
2. Configure your client, if it has the feature, to automatically follow HTTP 301 redirects. For example, if you use curl add the –location or -L option.

How does this site relate to the USGS Water Data for the Nation site (waterdata.usgs.gov)?

This site complements the USGS Water Data for the Nation site. The data are identical between the two sites. The USGS Water Data for the Nation site is designed for browsing and human ad-hoc inquiries of USGS water data. This site is designed to accommodate those interested in acquiring data to frequent machine processing. All services are designed for high availability, fast response and to support modern output formats.

How much traffic does this site receive?

Site web traffic varies from month to month but growth has been strong. In a typical month, fifteen to 20 million requests are made to USGS water services. The site typically ranks second in the rankings of most trafficked USGS web sites tracked within the USGS National Web infrastructure.

Which services are most popular?

Using October 2012-September 2013 as a baseline, 86% of traffic was for the instantaneous values web service, 8% for the daily values web service, 5% for the site service and 1% for the groundwater levels service. It is not surprising that real-time data would be of the most interest.

What are the motivations for this site?

The site serves these major purposes:

1. It eases the acquisition of approved and provisional data from the USGS National Water Information System for the millions of users of USGS water data. The USGS hopes that this will spur both internal and external communities to use the data for any intended usage by allowing it to be easily acquired from a reliable, trusted and highly available source.
2. It allows the USGS Water Data for the Nation site to evolve. Over time, the USGS Water Data for the Nation site will be overhauled. It will use the same web services available to anyone to acquire its data. In the long run it will make that system easier to maintain and retrofit for new uses.

7 - Writing Fault Resistant Code

Writing programs to access USGS web services is usually straightforward. However, it can be easy to write a program that may later stop working if the USGS changes the service. You can reduce the likelihood of this happening to you by adopting some or all of our suggestions for writing fault-resistant code.

Join the Water Data System Notification Service

Most changes to services are anticipated to be minor, which means that well written programs should not break when new versions are introduced. Occasionally, a major change that is likely to break a program may have to be introduced. The USGS tries to give advance notification and when possible provide examples of data in a new format so developers can avoid having their applications break. However, this is only possible if we can communicate with you. To stay informed, you are advised to join the USGS Water Data for the Nation Notification List . We won’t spam you and you should receive relatively few but important emails of significant system events or upgrades.

Check HTTP error codes

Since this system uses Hypertext Transfer Protocol (HTTP), any application errors are reported in the HTTP headers. This means that when writing applications, it is important to first examine the HTTP status code that is returned in the HTTP response. The application server will return the error code along with a message describing the error in the event there is a problem. Programmers should always check the HTTP response code and if not a 200 handle the response as an exception. Among the status codes you may see:

• 200 OK - The request was successfully executed.
• 400 Bad Request - This often occurs if the URL arguments are inconsistent, for example in the instantaneous values service using startDT and endDT with the period argument. An accompanying error should describe why the request was bad.
• 403 Access Forbidden - This should only occur if for some reason the USGS has blocked your Internet Protocol (IP) address from using the service. This can happen if we believe that your use of the service is so excessive that it is seriously impacting others using the service. To get unblocked, send us the URL you are using along with the IP using this form . We may require changes to your query and frequency of use in order to give you access to the service again.
• 404 Not Found - Returned if and only if the query expresses a combination of elements where data do not exist. For multi-site queries, if any data are found, it is returned for those site/parameters/date ranges where there are data.
• 500 Internal Server Error - If you see this, it means there is a problem with the web service itself. It usually means the application server is down unexpectedly. This could be caused by a host of conditions but changing your query will not solve this problem. The application support team has to fix it. Most of these errors are quickly detected and the support team is notified if they occur.
• 503 Service Unavailable - The application server is working but this application is down at the moment. When something causes this to happen, the support team should be quickly notified. Hopefully the service will be available shortly.

If your application is server-based, acquire data in XML if the format exists

Most services offer data in a Extensible Markup Language (XML). XML was written specifically to minimize issues associated with changes to data formats (which is why it is called extensible). If your application is correctly written, new tags and attributes to XML data should not cause your application to fail.

We recommend you acquire data in XML for all server-based applications if it is available. For example, if you have programs written in PHP, Perl, Python, Java, JSP or ASP that collect then process and/or re-serve USGS data, you are strongly encouraged to retrieve data in a XML format. You should avoid processing data in tab-delimited (RDB) or Excel formats specifically because you are likely to eventually have the application break when the content of the data changes.

If your application is browser-based or client-based (such as a native app), acquire data in the JSON format if available

JSON (Javascript Object Notation) is a compact way of acquiring data for a service optimized to be consumed by asynchronous Javascript supported by modern browsers. All services support the Cross-Origin Resource Sharing (CORS) specification. If your browser supports CORS the data can be directly acquired by the browser without introducing a browser security extension.

If a service offers a version number, request that version of the service

If you specify a version number for a data format, as long as it is supported it is unlikely to have its structure changed. As an example, the USGS Instantaneous Values REST Web Service offers a WaterML 1.1 version of the service. You can specify either the most current version of WaterML (the default) or a specific version that is still supported. So rather than create a URL like this:

https://waterservices.usgs.gov/nwis/iv?format=waterml&sites=01646500&parameterCd=00060,00065

why not specify the version as well, like this?

https://waterservices.usgs.gov/nwis/iv?format=waterml,1.1&sites=01646500&parameterCd=00060,00065

Check the service format syntax to see if it supports a version.

Write your queries efficiently

Here are some easy ways to get the data you are interested in efficiently:

• If regularly polling for certain site numbers, retrieve them in one query, by including them in the sites parameter. Separate site numbers with commas. Ex:

  https://waterservices.usgs.gov/nwis/iv/?format=waterml,2.0&sites=01646500,01638500&parameterCd=00060,00065

• Similarly, if you need certain parameters from each site of interest, separate the parameter codes with commas. If the site does not serve a particular parameter, no error will occur. The parameter simply will not appear in the output for that site. See the previous example, which will look for 00060 (discharge) and 00065 (gage height) at both sites, even if not present.

• If you keep a local cache of data, don’t poll for the same data over and over again. Instead, poll for new or changed data using the modifiedSince argument. See each service description for examples of how this is used.

• For instantaneous values, polling hourly is usually sufficient, since typically a site transmits only once an hour when it is permitted to broadcast to broadcast to the satellite.

• For daily values, polling once a day is sufficient as daily values are typically computed once a day.

• When using the statistics service, bear in mind that historical statistics rarely if ever change. The same query should never need to be rerun.

• Site information rarely changes, so if polling the site service to see if site information has changed, once a month is more than adequate. Certain period of record information changes daily if you need that detail, if the site is actively collecting time-series data.

Warning: if the USGS determines that your usage is excessive, your IP(s) may be blocked. If you get a HTTP_403 (Access Forbidden) error, you have likely been blocked. We may require you rewrite your queries or poll less frequently to use the service. If this happens to you, contact us .

Why you should avoid tab-delimited (RDB) files

It’s not always possible to avoid retrieving data in a tab-delimited format, since it is a legacy format and still useful. However, using it can be dangerous. For example, in November 2009, the USGS Water Data for the Nation site introduced a time zone column into many of its tab-delimited (RDB) files. Some of the changes introduced are shown below in bold:

agency_cd site_no datetime **tz_cd** 07_00065 07_00065_cd 02_00060 02_00060_cd
5s 15s 16d **6s** 14n 10s 14n 10s
USGS 06130500 2009-10-29 00:00 **MDT** 2.35 163
USGS 06130500 2009-10-29 00:15 **MDT** 2.35 163

This change allowed measurement times to be reported more accurately. However, some users who were not prepared for the change had their programs stop working. Previously, the fourth column held a number. Now it contains characters indicating the time zone.

While it is relatively simple to write a program that parses tab-delimited text and hard-code assumptions like the fourth column will always contain a number, when changes are introduced it can cause an application to fail. This is why the USGS recommends acquiring data in XML format instead of tab-delimited (RDB) format.

Parse XML using an XML parser

Almost all programming languages offer an XML parser. An XML parser is a tool that allows data inside an XML structure to be more easily manipulated. However, some programmers may prefer to use simple logic, such as reading data into an array, rather than use a parser. This is not recommended and could cause your application to break. While it does take a little effort to learn to use a parser, the effort invested pays many dividends, including making your program less likely to break if the schema changes.

Consider this simple XML document:

<?xml version="1.0"?>
<note>
  <to>Tove</to>
  <from>Jani</from>
  <heading>Reminder</heading>
  <body>Don't forget me this weekend!</body>
</note>

One way to get the data would be to read each line into an array. If you needed to capture the information in thetag, you could just read the sixth element of the array and strip off the tags. This would work fine until one day someone changed the schema to this:

<?xml version="1.0"?>
<note>
  <to>Tove</to>
  <from>Jani</from>
  <heading>Reminder</heading>
  <priority>Normal</priority>
  <body>Don't forget me this weekend!</body>
</note>

At this point your script would return incorrect information. While you could certainly examine all elements of your array for the <body> tag, parsers will typically put the data into a hierarchy automatically. Parsers also often have methods allowing an XML hierarchy to be searched.

Use standard libraries

Avoid reinventing the wheel. Regardless of your programming language, there is probably an off the shelf library available if the data you are trying to manipulate is in a standardized format. For example, WaterML expressed dates and times in the ISO-8601 standard. If you are a Java developer, the Joda Time API makes it easy to reliably parse these date/time values. Perl and PHP have similar libraries. For browser-based applications, jQuery and Dojo are among two of a number of popular Javascript frameworks that offer rich sets of features that are easy to integrate and will make your application look more professional.

Other tips

Consider using curl or wget to acquire data

For UNIX or Linux based systems, curl or wget may be available. wget is also available for Windows. These utilities allow sophisticated methods for acquiring data over the internet. Some programming languages have integrations with curl and wget. For example, PHP has an integrated curl library.

Use scheduled tasks to automate data collection

There are numerous ways to automate the downloading of data. Most operating systems come with the ability to automatically perform scheduled tasks at regular intervals. If your computer runs either Linux or some variant of Unix, the cron utility will be of interest. Windows has a task scheduler . You can use the appropriate utility to run your program.

Guidance on how often you should fetch data

Most USGS water data changes infrequently. Consequently, there is little need to fetch the same information repeatedly. The USGS monitors usage of its servers and if it detects users who are egregiously acquiring infrequently changing data, it may block service so others users are not impacted.

• Instantaneous values are updated generally once an hour (sometimes less often) for a real-time site. Typically, recent instantaneous values become available and older values although typically marked as provisional are not manually corrected. Do not fetch the same data more than hourly. We encourage use of the modifiedSince feature of the service which offers a “send me the data only if something has changed” feature and using a local cache otherwise.
• Daily values change rarely. Daily values are computations based on time-series measurements for a particular day and for a particular parameter. A daily value is only recomputed if a USGS water science center has corrected one or more of the time-series values for a given day. We recommend checking to see if old daily values have changed for a particular site no more than monthly.
• Site data changes very rarely. We recommend acquiring a range of sites no more than monthly.
• Statistics are based on daily values. They should change only if a USGS water science center approves provisional data. This is done irregularly, so follow the guidance for daily values.

Warning on provisional data

Much of the time-series data provided by these web services are provisional. In most cases, these data are accurate, but some measurements or calculations may not be accurate. This is particularly true of recently acquired data. These errant “spikes” are often later corrected during a formal review. When time-series data are acquired in WaterML (an XML format) each measurement is marked with an attribute indicating whether the data are provisional or not.

You should be careful to qualify provisional data as provisional and treat provisional data as potentially erroneous. Guidance on USGS provisional data.

Questions?

If you have any questions, use this form .