Category:Historical Documents Working Group

Overview

In the Linked Earth context, a working group (WG) is a self-organized coalition of knowledgeable experts, whose activities are governed herewith. This page is dedicated to the discussion of data and metadata standards for historical documents, and aims to formulate a set of recommendations for such a standard. This working group is an initiative of the Pages Floods Working Group but also related to the Pages CRIAS Working Group.

Sources

Data is usually compiled from different historical sources. The LiPD data structure supports several Publication, that is normally used for referring to the publication describing the data. So this data cluster can be used for historical sources as well, in addition to the current publication describing the data. It is related to known standards as Dublin Core or BibTEX

• Source-ID for later reference
• Source Type (string), i.e. newspaper, book, ...
• Source Author
• Source Title
• Publication Year
• Publication Date
• Journal Title
• Publisher
• Url to Source (i.e. to PDF)
• DOI of Source (almost never exist for historical documents if not published as data compilation)

• Source

The source type is a string i.e. book, newspaper, diary, ...

For details on possible options see Enumerator Options

Scans, Pages

Each source can optionally have a bunch of images, that are the scans of the pages inside the source. Maybe this can be dropped for the LiPD format and only references to external images should be added to the quotes.

Quotes

Out of each sources, several quotes could be extracted by transcribing them. Related data would most probably go into "measurementTables".

• Quote-ID: For later reference
• Reference to source: maybe short form like "Author()Year" is adequate
• Page: String of the page(s) number where the quote is extracted from
• Scan: Optional link(s) to externally stored image of this page(s)
• Language: The language of the quote (or it's translation)
• Protolanguage: The language the quote was written originally
• Quote: The quote itself (UTF8)
• License: License of the quote (cc)
• DOI: DOI, the quote is published

• Quote

The language (and protolanguage) is a string i.e. de, ar, fr, en,...

For details on possible options see Enumerator Options

See i.e. Wikipedia Language Code

The license type is a string i.e. cc-by, cc-by-nc, ...

For details on possible options see Enumerator Options

Events

Events, that are more like interpretations of the quote would go into "model part". Each events refers to

• a quote
• a position
• a time

and contains a climate related data. (to be defined)

Position

The position different than for the other archive types is not fixed. Usually a compilation of several sources refer to different scattered locations. The location usually can be named but might covers different scales (continent, country, area, city, street,...) and terrain types (city, river, sea, ...). It best corresponds to the Geospatial metadata of LiPD; see also http://schema.org/Place .

• Position

The location type is a string i.e. country, city, river, mountain, ...

For details on possible options see Enumerator Options

Time

The time derived for an event from historical documents is usually more precise than for other archive types. Often the uncertainty is just days or even hours. So it would be best to code the absolute date by gregorian calendar and a string of type ISO 8601. Optionally an absolute gregorian year (may be with floating point fraction instead of integer) can be calculated as well.

• Time

The original sources often contain the information in calendar notation different than gregorian. Julian calendar was used in Europe in historical time, but outside Europe other calendars existed or still exist, likeweise in chinese or arabic documents.

The calendar type is a string i.e. country, city, river, mountain, ...

For details on possible options see Enumerator Options

If so, which parameters for start and end should be considered?

It is also possible to extract the time related text from the quote and add it to a field 'phrase':

Event itself

The event refers to the quote, maybe directly to the source, to the chronology/timing and the position/location. It also holds the coding of the phenomena, here it gets complicated - see next section.

The license type is a string i.e. cc-by, cc-by-nc, ...

For details on possible options see Enumerator Options

Phenomena/Coding

A lot of information can be found inside historical documents. Thereby the coding schema is complex. A lot of coding is done by using indices. Some examples and their coding to illustrate this:

• The weather was extremely warm -> temperature index + very hot (+3)
• A lot of rain fell in December -> precipitation type + rain & precipitation amount + more than usual (+1)
• The water level was 13 feet -> water level + value:13.0 + tolerance:1.0 + unit:feet
• The drought lead to bad harvest of potatoes -> harvest + harvest amount: less than usual (-1) & potatoes / precipitation amount: less than usual (-1)
• The wheat harvest was little but the wine quality was good -> harvest + harvest amount: less than usual (-1) & wheat / harvest + harvest quality: better than usual (+1) & wine

+: one dimensional combination

&: two (or more) dimensional combinations

/: two different events (i.e. cause vs effect)

(Coding tree as used in tambora.org can be found here.)

To handle the innumerable amount of different parameters, it makes sense to group them into thematic clusters.

Cluster: Temperature

Inside this cluster are all temperature related parameters: Temperature measurements, descriptive temperature levels, freezing events, ...

• temperature-cluster

Levels i.e. can be

-3: very cold
-2: cold
-1: cool
 0: normal temperature
+1: warm
+2: hot
+3: very hot

Trend i.e. can be

-1: decreasing
 0: constant
+1: increasing

Temperature value is a measured, numerical value with units Standards of other archive types can be used here

The temperaure type is a string i.e. air, water, wind, ... It can be combined with the above quantitative parameters

For details on possible options see Enumerator Options

The following two parameters are of type boolean - they happen or not.

Cluster: Precipitation

Inside this cluster are all precipitation related parameters: long- and short-term amounts, intensities, measurements, type of precipitation, ...

• precipitation-cluster

Levels of long-term precipitation (weeks... month) i.e. can be

-3: extremely dry
-2: very dry
-1: dry
 0: normal precipitation
+1: wet
+2: very wet
+3: extremely wet

Levels of short-term precipitation (hours ... days) i.e. can be

 0: no precipitation
+1: some precipitation
+2: much precipitation
+3: very much precipitation
+4: extremely much precipitation

The amount of precipitation is a measured value for a given time frame (see timing)

The intensity of precipitation is a measured value, often a peal value

The precipitation type is a string i.e. rain, snow, hail, dew, sleet, ... It can be combined with the above quantitative parameters, but can also stand alone

For details on possible options see Enumerator Options

Levels of snow cover i.e. can be

 0: no snow cover
+1: thin snow cover
+2: deep snow cover

The snow depth is static (accumulated), whereas the snow fall is per time-frame

Cluster: Hydrology

see also Floods Working Group

Inside this cluster are all hydrology related parameters: water levels, flood magnitudes, ...

• hydrology-cluster

Flood levels (magnitude) i.e. can be

 0: no flood
+1: small flood
+2: flood above average
+3: flood of the century

Flood extend i.e. can be

+1: regional
+2: supra-regional

low water levels i.e. can be

 0: normal level
-1: low water level (limited use)
-2: very low water level (severely limited use)
-3: extremely low water level (no water)

Trend i.e. can be

-1: decreasing
 0: constant
+1: increasing

Storm surge levels i.e. can be

 0: no storm surge
+1: average storm surge
+2: strong storm surge
+3: extreme storm surge

Following parameters are of type boolean

Cluster: Clouds, Visibility

Inside this cluster are all cloud and visibility related parameters: cloud types, cloud cover, humidity, fog, ...

• cloud-visibility-cluster

Levels i.e. can be

 0:   0% sunny clear
+1:  25% mostly sunny
+2:  50% partly cloudy
+3:  75% very cloudy
+4: 100% fully cloudy

The cloud type is a string i.e. cumulus, nimbostratus, cirrus, ...

For details on possible options see Enumerator Options

Levels of humidity level i.e. can be

-1: dry air
 0: moderate humid air
+1: humid air

Levels of visibility i.e. can be

-2: very poor visibility
-1: poor visibility
 0: average visibility
+1: good visibility
+2: very good visibility

Following parameters are of type boolean

Cluster: Wind, Air Pressure

• wind-pressure-cluster

wind force levels (Beaufort) i.e. can be

  0: calm
 +1: light air
 +2: light breeze
 +3: gentle breeze
 +4: moderate breeze
 +5: fresh breeze
 +6: strong wind
 +7: high wind
 +8: fresh gale
 +9: strong gale
+10: storm
+11: violent storm
+12: hurricane

wind directions i.e. can be

  0: north
 +1: north-east
 +2: east
 +3: south-east
 +4: south
 +5: south-west
 +6: west
 +7: north-west
 +8: change of direction
 +9: any direction

The type of wind is a string i.e. foehn, gust, khamsin, sandstorm, sirocco, snowstorm, tornado, ... It can be combined with the above quantitative parameters, but can also stand alone

For details on possible options see Enumerator Options

air pressure levels i.e. can be

-1: low pressure
 0: normal pressure
+1: high pressure

air pressure trend i.e. can be

-1: falling pressure
 0: steady pressure
+1: raising pressure

Cluster: Society

Inside this cluster are all parameters related to society affections: damage, death, illness, social problems

• society-cluster

The type of damage is a string i.e. animals, buildings, bridges, houses, dams, plants, forest, traffic routes, ships, harvest, hunger, thirst, ...

For details on possible options see Enumerator Options

Illness is a string i.e. bubonic plague, fever, measles, cholera, ...

For details on possible options see Enumerator Options

Social problems is a string i.e. migration, poverty, rioting, theft, unemployment ...

For details on possible options see Enumerator Options

Levels of affected people i.e. can be

 0: none
+1: few
+2: some
+3: many

Cluster: Plant Phenology

Inside this cluster are all parameters related to plants: blossom, leave fall, harvest, ...

• plant-phenology-cluster

Levels of harvest quantity i.e. can be

-2: very low harvest volume
-1: low harvest volume
 0: average harvest volume
+1: high harvest volume
+2: very high harvest volume

Levels of harvest quality i.e. can be

-2: very poor crop quality
-1: poor crop quality
 0: average crop quality
+1: good crop quality
+2: very good crop quality

The plant type is a string i.e. apples, wheat, hay, wine, potato, cherries, ... It can be combined with the above quantitative parameters or the below observations, but can also stand alone

The type of observation is a string i.e. blossom, harvest, leave emergence, leave fall, planting, sowing, ...

Levels timing i.e. can be

-1: earlier than usual
 0: usual timing
+1: later than usual

Cluster: Animal Phenology

Inside this cluster are all parameters related to animals: health, arrival, departure, breeding, ...

• animal-phenology-cluster

Levels of absolute quantity i.e. can be

 0: none
+1: few
+2: some
+3: many

Levels of relative quantity i.e. can be

-1: less than usual
 0: average number
+1: more than usual

The animal type is a string i.e. birds, stork, sheep, swine, snail, locusts, ... It can be combined with the above quantitative parameters or the below observations, but can also stand alone

The type of observation is a string i.e. arrival, departure, breeding, shearing, slaughtering, death, injuries, ...

Levels timing i.e. can be

-1: earlier than usual
 0: usual timing
+1: later than usual

Cluster: Economy

Inside this cluster are all parameters related to economy: price-, supply-, demand- / -level, -trend, goods

• economy-cluster

Price level i.e. can be

-1: low price
 0: normal price
+1: high price

Price trend i.e. can be

-1: decreasing price
 0: constant price
+1: increasing price

Supply level i.e. can be

-1: low supply
 0: normal supply
+1: high supply

Supply trend i.e. can be

-1: decreasing supply
 0: constant supply
+1: increasing supply

Demand level i.e. can be

-1: low demand
 0: normal demand
+1: high demand

Demand trend i.e. can be

-1: decreasing demand
 0: constant demand
+1: increasing demand

The market good is a string i.e. apples, wheat, meat, food, energy, fodder, hay, wood, ... It can be combined with the above quantitative parameters

Cluster: Phenomena, Natural Hazards

Inside this cluster are all parameters related to natural phenomena or hazards: earthquake, tsunami, aurora borealis, meteoroid, comet, solar eclipse, moon eclipse, halo, avalanche, rainbow, landslide, wildfire, ...

• phenomena-hazard-cluster

The phenomena type is a string i.e. earthquake, tsunami, aurora borealis, meteoroid, comet, solar eclipse, moon eclipse, halo, avalanche, rainbow, landslide, wildfire, ...

Example

Rename LiPD file extension from zip to lpd

Description	Tambora-Files	LiPD-Files	Remarks
Flood Example	Media:flood_tambora_csv.zip	Media:Exp0000.tambora.2017.zip
To-Do	To-Do	To-Do
To-Do	To-Do	To-Do