WMTS time-query enhancements

By CubeWerx, Testbed-12 LSA Coverages NA006, 2016-10-07 draft.

Contact: Dr. Craig S. Bruce, csbruce@cubewerx.com

Introduction

This proposal extends the existing WMTS 1.0.0 and WMS 1.3.0 time-query mechanisms to provide general time filtering for general “scattershot” source data in a tiled environment. Scattershot source data is not pre-composited or mosiacked but is available in discrete, randomly-overlapping scenes that are time-stamped as originally captured (though are normally orthorectified). General time filtering needs to support client-selected time intervals, not just time instants. Enabling fully-arbitrary time intervals fulfills the needs of general time filtering, but is impractical to use with WMTSes that are based on pre-generated tiles because of the combinatorial explosion of possible min and max times for an interval, so simpler semantics are also needed. Tiles can be used for efficient WMS-Map rendering as well.

Eight useful semantics for time-based access are defined: ‘at’ time instant (as already in existing WMS/WMTS), ‘as-of’ time instant, fixed time-series, arbitrary time interval, and animated versions of the previous four. The arbitrary-interval semantic includes the capabilities of the simpler semantics, but it is important to allow simple server instances to provide limited, optimized service.

The WMTS Capabilities document reports precisely what times are available by giving a list of time semantics and start/end time intervals and resolutions with continuousness, time-granularity, and interval-bound-inclusiveness taken care of. ISO 8601 time-value encoding is used and it is important that time values be character-for-character-precise for accessing static, pre-generated tiles through the WMTS REST interface.

[The major section number of “1” is used throughout this document to allow the AsciiDoc content to be more easily included in other, larger-scope documents generated by the Testbed-12 project.]

Existing OWS time support

Time support exists in both the WMTS and WMS interfaces.

Existing WMTS time support

The latest official version of WMTS (Web Map Tile Service) is 1.0.0 [WMTS 1.0.0]. It includes a limited mechanism for dimensional queries including the dimension of time. An example XML Capabilities document might include:

<Dimension>
  <ows:Identifier>Time</ows:Identifier>
  <ows:UOM>ISO8601</ows:UOM>
  <ows:Title>Time</ows:Title>
  <ows:Abstract>Monthly datasets</ows:Abstract>
  <Default>null</Default>
  <Value>2007-05</Value>
  <Value>2007-06</Value>
  <Value>2007-07</Value>
</Dimension>

Specific values must be explicitly enumerated, which is not a scalable approach. If data were available on a daily basis for 2000 through 2015, there would need to be 5,844 enumerated Value tags. Alternatively, if a time-interval notation with a resolution of one day were available, only a single Value would need to be given. Time values are opaque in that the client does not parse them but simply parrots them back to the server. This effectively makes the values format-independent, but also means that clients cannot reinterpret them into into specialized calendar-based GUI components, for instance.

With Key-value-pair requests, the named Dimension identifier and selected value are included along with the standard request parameters. REST tile requests are indicated in the Capabilities document using templates with substitution variables in curly braces, such as:

<ResourceURL format="image/png" resourceType="tile" template="https://website.org/wmts/mylayer/{Style}/{Time}/{TileMatrixSet}/{TileMatrix}/{TileRow}/{TileCol}.png"/>

Existing WMS time support

The latest official version of the WMS (Web Map Service) is 1.3.0 [WMS 1.3.0]. It has more-sophisticated time support than WMTS. An example declaration section might be:

<Dimension name="Time" units="ISO8601" default="null">1996-01-01/2003-10-17/P1D</Dimension>

Unlike with WMTS, the WMS declaration is stuffed into a single XML tag with attributes and uses time intervals with declared resolutions, making them much more scalable but also requiring viewer clients to parse the values and perform date calculations. For multiple intervals, WMS adds a syntax of using a comma separator (“,”) between allowed time ranges as opposed to the WMTS method of using multiple Value fields. Time-instant values are represented using ISO 8601 [ISO 8601] notation.

The WMS data model for time is pretty clearly for pre-composited/mosaicked full-coverage source data that is neatly stacked by discrete if not regular time instants. The ‘nearestValue’ mechanism assumes that there is a single nearest time instant, as confirmed by its Warning message in the HTTP header.

The ‘multipleValues’ attribute distinguishes whether a list of values or only a single value may be requested. It is not clear whether a time interval qualifies as a single value or as multiple values, but Example 2 in section C.3.6 implies that an interval is considered to be multiple values suitable for an animation response.

WMS appears to offer only one base time-filtering semantic, which can be called “at”, or a variant from using the ‘nearestValue’ attribute that can be called “nearest-to”, plus animated versions of the previous two using the ‘multipleValues’ attribute. Only one of “at” or “nearest-to” can be in force in a declaration and the selection of animation depends on at least one factor outside of the scope of the time declaration (the data format for the result map).

It is not clear what happens if an interval request is made when ‘multipleValues’ is not supported by a server. Also, an interval request requires a resolution according to Table C.2, implying a “multiple-at” semantic rather than a true interval. It is not clear if the resolution in a request is allowed to be “0” as described immediately after the table (though “0” is not a valid ISO-8601 period syntax; “PT0S” would be valid). Section C.3.2 gives the template “TIME=start_time/current”, but this is not a valid syntax according to Table C.2 which requires a resolution.

In summary, neither WMS nor WMTS provides adequate support for general time filtering of general source data. Also, the exact WMS time semantics are not explained clearly enough.

Time-query semantics and encodings

Data environment

The source-data environment envisioned is a fully general “scattershot” one where the source data is contained in “scenes” such as satellite or aerial images that can have various resolutions, locations, coordinate systems, acquisition times, and “color” types (e.g., RGB, Color Palette, RGBI, numeric), sample sizes, and data formats and can overlap each other in arbitrary ways and can have large gaps between coverage areas. Interfaces like WMTS hide the source-data semantics, though these semantics have implications for the exact tile content retrieved.

Timeless queries and overlapping-content ordering

A time-capable web service should respond intelligently to queries that do not filter by time. One method would be to return content with the data with the latest acquisition time on top, though other ordering methods might override time in cases where some source data has a significantly finer resolution that other data or has a higher user-assigned priority. In principle, time-based queries could also be rendered in some order other than latest-time-on-top where source scenes overlap, if suitable for a service.

There should be an obvious way to request data without special time filtering, such as requesting the default value for a time dimension, using a timeless REST template, or having the interface define special semantics for the time parameters.

Time-value encoding basis

The basis for representing time values and semantics here is ISO 8601 notation, as in WMS 1.3.0. ISO provides several different encodings for the same time instant, but for an interface like WMTS REST, it is important to specify exact, character-for-character encodings. One approach would be to allow servers to specify the encoding of all of the components of a time value using a formatting template in order to accommodate prepared static data that has a pre-determined format, but this would probably not be a worthwhile exercise. Clients would need to be significantly more complicated to recognize such formatting and compose proper time values. Prescribing a time format can complicate a server in that it may need to provide a mapping between internal and external time-value representations but allows the client to be statically programmed.

The prickly issues of time zones and time granularity must also be dealt with for both source data and open interfaces. If a source time value is merely “2015-03-17”, precisely what time is meant by that? Midnight? Noon? The whole day? In what time zone or zones? To avoid ambiguity and simplify coding, all time values passed in the enhanced WMTS interface shall be in the UTC time zone and the encoding shall include the explicit ISO designation character of “Z” (Zulu) where applicable according to the ISO-encoding rules to make this specific semantic extremely visible. Using arbitrary time zones would be an unnecessary complication and using an implicit time zone relative to the server is very awkward because of local Daylight Savings rules.

Time instant

A time instant shall have the ISO general form “YYYY-MM-DDThh:mm:ss.dddddddddZ”, where the time components are Year, Month, Day, Hour, Minute, Second, and sub-second Decimals, using as many components, decimals, and punctuation characters as needed to encode times to a selected granularity. The “-”, “T”, “:”, “.”, and “Z” characters are literal. The time value shall end with the ISO “Z” character whenever the Hour (or finer) time component is included. The ISO comma (“,”) character is disallowed for the decimal point since exact coding will be needed for the WMTS REST interface and there is already an incompatible use for the comma character in the WMS Capabilities document. ISO also provides a mechanism to use years outside the range of 0001 to 9999 that is compatible with the approach used here.

In principle, sub-second precision could extend indefinitely, but the finest practical accuracy even for synchronizing atomic clocks is around one nanosecond. Choosing a granularity limit of less than a nanosecond is generally a mistake, since systems far too often go through the evolutionary cycle of selecting seconds, then upgrading to milliseconds out of necessity, then microseconds, and finally nanoseconds, because common computers can produce that amount of resolution (though not necessarily that amount of accuracy), though GPS synchronization can produce around 14 ns of accuracy in principle and 100 ns in practise. Just look at the evolution of the POSIX or SQL interfaces to see compatibility problems caused by choosing successively finer time granularities. The smartest move is just to start at nanosecond resolution and be done with it. A nanosecond within a second can be conveniently represented inside a system using a 32-bit integer.

Time granularity

The server shall identify the granularity of time values it will accept for character-for-character REST encodings with each specification of allowed time values using an integer to represent the number of time components and sub-second decimals. The following table gives the legal granularity values:

Granularity Integer Time Granularity

0

Any legal granularity 1–15 (special wildcard)

1

Year

2

Month

3

Day

4

Hour

5

Minute

6

Second

7

0.1 Second / Decisecond

8

0.01 Second / Centisecond

9

0.001 Second / Millisecond

10

0.000 1 Second / Hectomicrosecond

11

0.000 01 Second / Decamicrosecond

12

0.000 001 Second / Microsecond

13

0.000 000 1 Second / Hectonanosecond

14

0.000 000 01 Second / Decananosecond

15

0.000 000 001 Second / Nanosecond

The server and client are free to produce time values encoded to any legal granularity except where exact encoding to the indicated granularity is required by a REST interface. If the server specifies the wildcard granularity (code 0), then the client is free to use any legal granularity in all requests including with all REST interfaces.

It is useful to have a specific limit to granularity precision so that inclusive and exclusive time boundaries can be expressed unambiguously. This limit shall be the nanosecond. For example, the last legal time instant before “2015-03-17T00:00:00.000000000Z” is “2015-03-16T23:59:59.999999999Z”, so this value can be used explicitly as an inclusive upper limit to express a time “less than” or excluding “2015-03-17T00:00:00.000000000Z”.

All time instants expressed to any granularity in the interface shall be interpreted internally by both client and server as representing the given value expanded to its first nanosecond. For example, “2015-03-17” would be interpreted as “2015-03-17T00:00:00.000000000Z”. Time instants re-expressed to a lesser granularity are always truncated instead of rounded. For example, “2015-03-16T23:59:59.999999999Z” re-expressed to Second granularity is “2015-03-17T23:59:59Z”. (One would not round up 10:45 to say that it is 11 o’clock.)

Time interval

A time interval is specified using the inclusive first time instant and the inclusive last time instant, later than or equal to the first time instant, separated by the slash (“/”) character. Both time instants are interpreted as being expanded to their first nanosecond. For example, “2013-01-01T17Z/2015-03-17T17Z” would be interpreted as the inclusive interval from “2013-01-01T17:00:00.000000000Z” to “2015-03-17T17:00:00.000000000Z”. The two given time instants can be expressed to different granularities except where precise encoding is required by a REST interface. Be careful to give the appropriate ending instant recognizing that it is inclusive. Also beware of unintentional overlap in a list of time intervals that are not meant to overlap.

ISO also allows either or both time instants to be empty strings corresponding to positive and negative infinity, but this notation is not needed here as explicit interval starting and ending times can always be given unless a layer has no time information available at all (perhaps because it is completely empty of content). In such a case, the server has various options, such as reporting a special null-value as an acceptable old-style time value that it knows how to interpret.

For comparisons, using the nanosecond granularity limit makes it possible to effectively indicate whether time boundaries are inclusive or exclusive. For example, if a time interval is to be specified from noon 2015-03-17 EDT to 2pm but excluding both bounds, a proper expression would be “2015-03-17T16:00:00.000000001Z/2015-03-17T17:59:59.999999999Z”. If the two bounds were inclusive, it could be expressed more compactly as “2015-03-17T16Z/2015-03-17T18Z”.

The slash (“/”) interval-separator character is very problematic for the WMTS REST interface because it is the folder separator used in most URI schemes and maps to the web-server file system, so using it would cause awkward interference with parsing pathnames. For this reason, in the WMTS REST-request URI only, two dash characters (“--”, U+002D U+002D) shall be used in place of the slash character when encoding interval time values and other time parameters like resolutions.

Time resolution

ISO provides an encoding for time repetitions, but it is needlessly awkward. WMS defines a simpler format for repeated time instants/resolutions/intervals: “first_instant/last_instant/period”, using an ISO period as the last component, which is used here.

The period component has the general format “P[yY][mM]T[hH][mM][sS]”, where the square brackets indicate each optional component. The letter “P” is always present and means “Period”. The other components give the number of years, months, days, hours, minutes, and/or seconds, where each value can be any floating-point number. Note that different implementations are likely to interpret non-round results slightly differently. The letter “T” must always be used as a separator in the appropriate position if any time components to its right are used, as the letter “M” would otherwise be ambiguous. Examples are “P1D” for one day, “PT1H” for one hour, and “P1.5D” or “P1DT12H” for one-and-a-half days.

A time interval with a resolution identifies a series of discrete time instants defined by the starting time instant plus all whole-number multiples (0, 1, …) of the given resolution (period) earlier than or equal to the last time instant of the interval. (The “last” instant is a limit that is not necessarily included in the set of specified instants.) For example, “2010-01-05T17Z/2016-03-25T17Z/P14D” defines the first nanosecond of noon EST (disregarding Daylight Savings) of every second Tuesday in the time interval. The first time instant of the series is “2010-01-05T17Z”, the second, “2010-01-19T17Z”, and the last, “2016-03-22T17Z”. Note that precise time values in UTC do not necessarily align with time granularities, nor do resolutions.

If a time interval is given without a resolution to describe the times for which data is available, it means that any time instant (or interval) within the interval bounds can be requested (subject to any granularity restrictions).

It can also be useful to provide a resolution for a single time instant in order to distinguish it from data prepared for different time resolutions available at the same instant (e.g., daily, monthly, yearly relative to instant “2015-01-01”). The special notation “instant/resolution” is used where appropriate for this purpose.

Frames per second

For animations, an additional parameter can be appended to a time specification to request the number of frames per second to use for the animation. The format is “/Ffps”. The “fps” is a floating-point number. The default value is server-specific.

WMTS interface extensions

Time is added to the WMTS 1.0.0 interface using the existing Dimension metadata structure. The Capabilities document indicates precisely what times are available with a granularity indicator and a list of time semantics with start/end time intervals sometimes with time resolutions. ISO 8601 time-value encoding is used and it is important that time values be character-for-character-precise when used with static, pre-generated REST tiles. This allows a time-based WMTS to be implemented using only a plain web server with no special runtime component or by using a large-scale service like Amazon CloudFront to enable potentially millions of concurrent users.

Capabilities Dimension declaration

The structure of the WMTS 1.0.0 Dimension declaration is retained but with new values in the UOM and Value fields. This creates conflict between these field values which may necessitate a new version number for the WMTS to distinguish the semantics.

The UOM field (Unit Of Measure) shall identify and parameterize the granularity of a new time dimension and has the following format: “ISO8601/granularity”. The UOM “granularity” value is the numeric granularity level from 0 to 15 shown in the table in the “Time granularity” clause. It is important to explicitly express the time granularity to use with REST requests up front exactly one time in order to be clear and unambiguous. It is also useful to have a different UOM value from the WMTS 1.0.0 literal “ISO8601” to indicate for old clients that actually check that the enumerated values are to be handled differently from 1.0.0. Also, unless and until this specification is incorporated into the official WMTS standard, a different dimension name such as “QTime” (query time) should also be used to avoid compatibility problems with clients that are hard-wired to recognize the dimension name “Time” with the old encoding.

The time instants and time semantics available for querying are given in the Value fields of the Dimension. An example Dimension declaration is:

<Dimension>
  <ows:Identifier>QTime</ows:Identifier>
  <ows:UOM>ISO8601/3</ows:UOM>
  <ows:Title>Time</ows:Title>
  <ows:Abstract>blah blah blah</ows:Abstract>
  <Default>alltime</Default>
  <Value>at:2007-05-01/2007-07-31/P1D</Value>
  <Value>asof:2007-05-01/2007-07-31/P1D</Value>
  <Value>series:2007-05-01/2007-07-31/P1D</Value>
  <Value>interval:2007-05-01/2007-07-31</Value>
  <Value>anim.asof:2007-05-01/2007-07-31/P1D/F12</Value>
  <Value>anim.interval:2007-05-01/2007-07-31</Value>
</Dimension>

Time values with the new semantics shall be identified by special short prefix strings. The WMTS 1.0.0 opaque-value semantics are maintained for all values that do not start with a new-time-semantic prefix. The new prefix shall be followed by an interval with a usually optional resolution which identifies precisely what time instants are legal for querying. If a resolution is given, it restricts legal query instants to only those that are integer multiples of the resolution from the starting time; otherwise, any time instant within the interval is legal. With time semantics that use a query interval in the client request, only legal time instants can be used for the starting and ending times of the interval. Animation semantics can also include an optional frames-per-second parameter if it is fixed by the server. Specifically, the Capabilities-declaration format is “prefix:first_instant/last_instant[/resolution].

The new time-semantic Value-prefix strings for single-frame still images are:

at:

Selects a server-defined semantic for a single time instant. The WMTS 1.0.0 interface provides an interface to query tiles “at” specific server-defined time instants. The true meaning of the time instant is not clear, as it could be truly “at” a specific time instant, a time-series interval starting at or ending at the indicated time, data “as-of” a certain time, etc. Often, the precise semantic is unimportant or already known by the user. The server gives an accessible-time interval with an optional resolution; the client requests the sought time instant. The “at” client-query format is “at:instant”, for example, “at:2007-05-20”.

asof:

Selects all times from the first time instant up to and including the given time instant. The “as-of” time-value semantic is probably the most useful for casual viewing. It can be used as a “time machine” to examine how the coverage area looked according to the most recently available data as of any time in the past. A GUI control might be a slider bar covering the time range of the available data. The server gives an accessible-time interval with an optional resolution; the client requests the ending time instant. The “as-of” client-query format is “asof:instant”, for example, “asof:2007-05-20”.

series:

Selects one frame from a fixed time series identified by the first time instant of the frame. The resolution breaks the time interval into equal slices/frames (except perhaps for the last slice) and each slice includes its starting time and excludes the starting time of the next slice. The exact semantic of whether the time series defines instants or sub-intervals of time is server-dependent. Beware of the potential “fencepost problem” caused by the ending instant being inclusive. The server gives an accessible-time interval with a required resolution; the client requests a time instant plus the server-given resolution value (needed to disambiguate if multiple resolutions are available). In the case of a REST request, the resolution component shall be character-for-character exactly what the server gave. The time-series client-query format is “series:instant/resolution”, for example, “series:2007-05-20/P1D”. The implied query interval is from instant to instant + resolution − 1 nanosecond, modulo the data-availability interval.

interval:

Selects all times in a given arbitrary inclusive time interval. Normally, all source data that overlaps the time interval is selected for processing. The server gives an accessible-time interval with an optional resolution; the client requests a time interval. The time-series client-query format is “interval:first_instant/last_instant”, for example, “interval:2007-05-20/2007-06-08”.

Each still-image/single-frame method has an animated counterpart with a prefix-prefix of “anim.”. An animation is an orthogonal concept that selects multiple sequential frames of the base semantic with a time interval and at a specified resolution. The resulting content will normally be generated into a format that can hold multiple frames of animation, such as an MPEG4 video file, and with an optional given frame rate. Specifically, the Capabilities-declaration format is “prefix:first_instant/last_instant/resolution[/frames_per_second]”. All of the different semantics have the same query format, with only the meaning of the content being different (at, as-of, interval).

anim.at:

Selects an animation of “at” image frames. The server gives an accessible-time interval with a required resolution; the client requests a time interval with the same resolution. With a REST request, the resolution must character-for-character the same.

anim.asof:

Selects an animation “as-of” image frames. The server gives an accessible-time interval with a required resolution; the client requests a time interval with the same resolution. With a REST request, the resolution must character-for-character the same. An example client query time might be “anim.asof:2007-05-11/2007-07-08/P1D/F12”.

anim.series:

Selects an animation of fixed-time-series image frames. The server gives an accessible-time interval with a required resolution; the client requests a time interval with the same resolution. With a REST request, the resolution must character-for-character the same.

anim.interval:

Selects an animation of arbitrary-interval image frames with an arbitrary time resolution. The server gives an accessible-time interval with an optional resolution; the client requests a time interval with an arbitrary client-chosen time resolution that need not have any relation to the server-advertised resolution if there was one. An example client query time might be “anim.interval:2007-05-06/2007-07-28/P7D/F2”.

The client can request any legal time defined by any Value field of a Dimension or by the Default field. The client and server are free to generate time values to any legal granularity except in the case of a WMTS REST request, where the client must use the time granularity indicated by the server unless the wildcard granularity (code 0) is indicated. The REST granularity, the granularities in Capabilities, the granularities in non-REST requests, and the time resolutions in different Value fields may all be different.

As an example, a WMTS may offer an static-REST “as-of” time Dimension Value that overlaps new imagery scenes every second Tuesday but maintains a history of previous data versions. It might advertise its valid times as “asof:2010-01-05T12Z/2016-03-23T12Z/P14D” with a granularity of Hours (code 4). The first valid request time instant would be “2010-01-05T12Z”, the second “2010-01-19T12Z”, and the most recent, “2016-03-22T12Z”.

To help with the forward-compatibility of old clients, the Default field could specify a special value to request “timeless” or all-time tiles. Servers could also accept the raw REST template-variable encoding as selecting the default value. E.g., a REST request might “…/%7BQtime%7D/…”. The server could recognize that a client does not understand Dimension usage and select the default time value. It could also recognize the client parroting back the literal time values specified in the Dimension declaration if invalid as meaning that an old client does not know how to parse new time values, and react accordingly. Faulty JavaScript clients may generate the value “*undefined*” when substituting the variable and the WMS specification seems to also allow the value “*default*”.

Key-value request interface

A key-value client shall request times for tiles by using the Dimension Identifier as the key, selecting an appropriate Dimension Value field, and computing a legal time value according to the time semantic. The request value shall include the same time-semantic prefix as the Dimension Value.

An example request might be:

https://website.org?SERVICE=WMTS&REQUEST=GetTile&VERSION=1.0&LAYER=my_layer&STYLE=my_style&TILEMATRIXSET=smerc&TILEMATRIX=4&TILEROW=3&TILECOL=6&FORMAT=image/PNG&QTime=series:2007-06-04/P1D

Note that key-value parameter names are case-insensitive in OGC requests.

REST request interface

The REST method is specified in the Capabilities document using the ResourceURL template structure. The time key and value are computed similarly to key-value requests, and the value is substituted into the URI template variable with the same name as the key. For example, variable

There are three differences in generating the time values, however. The first difference is that the time instants shall be generated to the granularity indicated by the server in the UOM field. The second difference is that because of incompatibilities between REST URIs and the slash character used in ISO time intervals, etc., the slash character (“/”) in computed time values shall be substituted with two dash characters (“--”). The third difference is that with “series:”, “anim.at:”, “anim.asof:” and “anim.series:” time values, the time-resolution component shall be character-for-character exactly what the server indicated in the Dimension Value.

An example ResourceURL and REST-request URI are:

<ResourceURL format="image/png" resourceType="tile" template="https://website.org/wmts/my_layer/{Style}/{QTime}/{TileMatrixSet}/{TileMatrix}/{TileRow}/{TileCol}.png"
https://website.org/wmts/my_layer/my_style/series:2007-06-04-P1D/smerc/4/3/6.png

The special characters in the REST URI may be escaped with the “%xx” hexadecimal notation. This is compatible with character-for-character encoding.

References

[ISO 8601] Data elements and interchange formats — Information interchange — Representation of dates and times, https://en.wikipedia.org/wiki/ISO_8601

[WMS 1.3.0] OpenGIS Web Map Service (WMS) Implementation Specification, version 1.3.0, OGC document 06-042, 2006-03-15, http://portal.opengeospatial.org/files/?artifact_id=14416

[WMTS 1.0.0] OpenGIS Web Map Tile Service Implementation Standard, version 1.0.0, OGC document 07-057r7, 2010-04-06, http://portal.opengeospatial.org/files/?artifact_id=35326