Using LiPD files - Revision history

Michaelerb: /* LiPD Utilities in Python 3 */

2019-05-16T18:45:41Z

‎LiPD Utilities in Python 3

Michaelerb: /* LiPD Utilities in Python 3 */ Updated the commands so that they work again.

2018-08-14T16:50:05Z

‎LiPD Utilities in Python 3: Updated the commands so that they work again.

Michaelerb at 20:20, 3 July 2018

2018-07-03T20:20:18Z

Michaelerb: /* LiPD Utilities in Python 3 */

2017-06-27T17:00:35Z

‎LiPD Utilities in Python 3

Michaelerb: /* A Note for Windows Users */

2017-06-27T16:57:37Z

‎A Note for Windows Users

Khider: /* CSV files */ Add tutorial images

2017-06-27T16:27:47Z

‎CSV files: Add tutorial images

Khider: /* LiPD Utilities */ Add See Also

2017-06-27T16:16:37Z

‎LiPD Utilities: Add See Also

Khider: /* Pyleoclim */ Add Windows warning + See Also

2017-06-27T16:15:42Z

‎Pyleoclim: Add Windows warning + See Also

Khider: /* A Note for Windows Users */ Add Anaconda option

2017-06-27T16:13:05Z

‎A Note for Windows Users: Add Anaconda option

Michaelerb: /* CSV files */

2017-06-26T23:51:12Z

‎CSV files

@@ Line 55: / Line 55: @@
 </syntaxhighlight>
-A list of dictionaries is now contained in the variable "data_all".  (If you're unfamiliar with python, read a primer to get a better understanding.)  From here, the data can be sorted with the filterTs() function, according to metadata fields.  For example:
+An array of dictionaries is now contained in the variable "data_all_ts".  (If you're unfamiliar with python, read a primer to get a better understanding.)  From here, the data can be sorted with the filterTs() function, according to metadata fields.  For example:
 <syntaxhighlight lang="python">

@@ Line 42: / Line 42: @@
 <syntaxhighlight lang="python">
-lipd.readLipd('/path/to/data/file.lpd')  # Load a specific LiPD file.  Or...
+data_single   = lipd.readLipd('/path/to/data/file.lpd')  # Load a specific LiPD file.  Or...
-lipd.readLipd()                          # Load a LiPD file through a GUI.
+data_all      = lipd.readLipd('/path/to/data/')          # Load all LiPD files in a directory.  Or...
 </syntaxhighlight>
-After reading a LiPD file, use the extractTs() function to extract the data and metadata within the file:
+The data will will be stored in a dictionary, from which you can access all of the data.
 <syntaxhighlight lang="python">
-data_all = lipd.extractTs()
+data_all_ts = lipd.extractTs(data_all)
 </syntaxhighlight>
-A list of dictionaries is now contained in the variable "data_all".  (If you're unfamiliar with python, read a primer to get a better understanding.)  Each dictionary contains data and metadata fields.  From here, it's easy to start using the data:
+A list of dictionaries is now contained in the variable "data_all".  (If you're unfamiliar with python, read a primer to get a better understanding.)  From here, the data can be sorted with the filterTs() function, according to metadata fields.  For example:
 <syntaxhighlight lang="python">
-data = data_all[0]  # Save the first time series object to a new variable. Replace "0" with a different
+data = data_all_ts[0]  # Save the first time series object to a new variable. Replace "0" with a different
-                    # number to select a different time series object if there are more than one.
+                       # number to select a different time series object if there are more than one.
 print(data.keys())                 # Print the names of all data and metadata fields.
 print(data['dataSetName'])         # Print the contents of one field: the name of the data set.
@@ Line 63: / Line 74: @@
 </syntaxhighlight>
-Different data sets may not have all of the same fields, so use the ".keys()" command to check.  Now, let's make a simple figure:
+Now, let's make a simple figure:
 <syntaxhighlight lang="python">

@@ Line 33: / Line 33: @@
   pip install LiPD
-Start python with the command "python".  Then import lipd:
+If you run into any errors, they may indicate other things you need to install first.  Next, start python with the command "python".  Then import lipd:
 <syntaxhighlight lang="python">

@@ Line 75: / Line 75: @@
 </syntaxhighlight>
-From here, explore a LiPD file some more on your own.  There are other commands in the LiPD utilities, but the ones above are enough to access the data on a basic level.  If you'd like to use pre-built functions to explore the paleo data, see the Pyleoclim section farther down this page.
+From here, explore a LiPD file some more on your own.  There are other commands in the LiPD utilities, but the ones above are enough to access the data on a basic level.  To get started with some more commands, read the quickstart guide on Github: [https://github.com/nickmckay/LiPD-utilities/blob/master/Examples/Welcome%20LiPD%20-%20Quickstart.ipynb Quickstart Guide].
 ==Pyleoclim==

@@ Line 7: / Line 7: @@
 The LiPD utilities are compatible with Windows. However, most of the more advanced data analysis tools are only available for Mac or Linux platforms. If you're using Windows and would like to start using Linux, here are two options:
 # Ask your university for an account on their Linux machine.  Then use an ssh program like [http://www.putty.org/ PuTTY] to connect from your PC.  An X-windows program like [https://sourceforge.net/projects/xming/ Xming] is also needed to produce graphical windows.
-# If option one isn't a possibility, you could install a Linux virtual machine (e.g. [https://www.virtualbox.org/ VirtualBox]) on your PC.
+# If option one isn't a possibility, you could install an Ubuntu virtual machine (e.g. [https://www.virtualbox.org/ VirtualBox]) on your PC.
 If you insist on working within Windows, you can install Python on your PC directly from the [https://www.python.org/ Python website].  '''Make sure to select the Python 3 version'''.

← Older revision		Revision as of 16:27, 27 June 2017
Line 91:		Line 91:
	==CSV files==		==CSV files==

−	~~If you’re~~ in ~~a jam and~~ need a plaintext version of the data, all LiPD files contain .csv files of the raw data. Simply unzip your LiPD file to find a .csv file. However, a central goal of LiPD is to put paleoclimate data into a standardized format which common analysis scripts can be built for, so using .csv files more than necessary is not recommended. Also, re-zipping an unzipped LiPD file will not turn it into a valid LiPD file.	+	[[File:Example of Unzip LiPD file.png \| thumb \| right \|400px \| Example of an unzipped LiPD files. The data is contained in the csv files while the metadata is stored in the JSON-LD file]]
		+
		+	If you need a plaintext version of the data, all LiPD files contain .csv files of the raw data. Simply unzip your LiPD file to find a .csv file. However, a central goal of LiPD is to put paleoclimate data into a standardized format which common analysis scripts can be built for, so using .csv files more than necessary is not recommended. Also, re-zipping an unzipped LiPD file will not turn it into a valid LiPD file.
		+
		+	[[File:BestPractices Download csv file.png \| thumb \| right \| 400 px \| Downloading a csv file]]
		+
		+	You can also access the .csv file on the wiki and download them.

← Older revision		Revision as of 16:16, 27 June 2017
Line 20:		Line 20:

	==LiPD Utilities==		==LiPD Utilities==
		+
		+	{{See also\|LiPD Utilities}}

	The [[LiPD Utilities]] are a primary way to interact with LiPD files. The utilities are available on [https://github.com/nickmckay/LiPD-utilities GitHub] in Matlab, R, and Python language. All three languages support reading and writing a LiPD file, extracting and collapsing time series, and filtering time series.		The [[LiPD Utilities]] are a primary way to interact with LiPD files. The utilities are available on [https://github.com/nickmckay/LiPD-utilities GitHub] in Matlab, R, and Python language. All three languages support reading and writing a LiPD file, extracting and collapsing time series, and filtering time series.

@@ Line 76: / Line 76: @@
 ==Pyleoclim==
 [[Pyleoclim]] is another primary way of interacting with LiPD files.  While the LiPD Utilities offer a more manual approach to analysis, Pyleoclim has a variety of pre-built functions which can save time and effort.  Install pyleoclim with:
@@ Line 82: / Line 84: @@
 To get started with pyleoclim, see the [[Pyleoclim]] wiki page.  In particular, read the quickstart guide: [https://github.com/LinkedEarth/Pyleoclim_util/blob/master/Example/PyleoclimQuickstart.ipynb Quickstart Guide].
 ==CSV files==
 If you’re in a jam and need a plaintext version of the data, all LiPD files contain .csv files of the raw data.  Simply unzip your LiPD file to find a .csv file.  However, a central goal of LiPD is to put paleoclimate data into a standardized format which common analysis scripts can be built for, so using .csv files more than necessary is not recommended.  Also, re-zipping an unzipped LiPD file will not turn it into a valid LiPD file.

@@ Line 5: / Line 5: @@
 ==A Note for Windows Users==
-A Linux environment is recommended for data analysis.  Mac or Linux users should already have access to Linux, so they can skip the rest of this section.  If you're using Windows and would like to start using Linux, here are two options:
+The LiPD utilities are compatible with Windows. However, most of the more advanced data analysis tools are only available for Mac or Linux platforms. If you're using Windows and would like to start using Linux, here are two options:
 # Ask your university for an account on their Linux machine.  Then use an ssh program like [http://www.putty.org/ PuTTY] to connect from your PC.  An X-windows program like [https://sourceforge.net/projects/xming/ Xming] is also needed to produce graphical windows.
 # If option one isn't a possibility, you could install a Linux virtual machine (e.g. [https://www.virtualbox.org/ VirtualBox]) on your PC.
-If you insist on working within Windows, you can install Python on your PC: [https://www.python.org/ Python].  Make sure to select the Python 3 version.
+If you insist on working within Windows, you can install Python on your PC directly from the [https://www.python.org/ Python website].  '''Make sure to select the Python 3 version'''.
 After installing python, open a Windows command prompt.  Despite cosmetic similarities, a Windows command prompt is different from a Linux terminal and generally uses different commands.  To launch Python from here, type "python".  To install python packages in Windows, exit python and type:
@@ Line 16: / Line 16: @@
 where "package_name" is the name of a python package, such as LiPD.  However, not all packages may be available for Windows, so Linux is still recommended.
 ==LiPD Utilities==

← Older revision		Revision as of 23:51, 26 June 2017
Line 83:		Line 83:
	==CSV files==		==CSV files==

−	If you’re in a jam and need a plaintext version of the data, all LiPD files contain .csv files of the raw data. Simply unzip your LiPD file to find a .csv file. However, a central goal of LiPD is to put paleoclimate data into a standardized format which common analysis scripts can be built for, so using .csv files more than necessary is not recommended.	+	If you’re in a jam and need a plaintext version of the data, all LiPD files contain .csv files of the raw data. Simply unzip your LiPD file to find a .csv file. However, a central goal of LiPD is to put paleoclimate data into a standardized format which common analysis scripts can be built for, so using .csv files more than necessary is not recommended. Also, re-zipping an unzipped LiPD file will not turn it into a valid LiPD file.