NetCDF (network Common Data Form) is an interface set for array-oriented data access and a freely-distributed collection of data access libraries for C, Fortran, C++, Java, and other languages. It is a hierarchical data format similar to hdf4 and hdf5 (Hierarchical Data Formats). It is known as a “self-describing” data structure, meaning metadata, or descriptions of the data, are included in the file itself and can be parsed programmatically. That means they can be accessed using code to build automated, reproducible workflows.
NetCDF was developed to handle multidimensional data and is widely used in the geoscience education research community. It’s flexible and self-describing, which means that it contains a lot of information like the dimensions, the variables, and some other information related to the dataset. Each NetCDF file has its variables and dimensions, which differ from others.
The image below shows a sample of NetCDF data on the left used to plot the image on the right.
NetCDF data
What’s a CSV File?
A Comma Separated Value (CSV) file is a simple text file where data or values are separated using commas. The image below shows a CSV file with several addresses.
CSV File
Converting NetCDF files to CSV on Chromebook
This post will give you a detailed guide on Converting NetCDF files to CSV on Chromebook. When Google Chromebooks were first released in the market, most users were a little skeptical since it only came with a browser, a file manager, and a few Google applications. However, it received enormous updates after a while, enabling the device to run android and Linux applications.
We will leverage these new features to convert NetCDF files to CSV on a Chromebook. Most of the methods we will look at in this post require you to enable Linux on your Chromebook.
Enable Linux on Chromebook
Please check out our post How to install and configure Linux Apps on Chrome OS, which gives you a step-by-step guide on installing and running Linux apps on your Chromebook. Luckily, the process itself is not complicated. First, launch the Settings application and click on the Linux (beta) option on the left side. Next, click “Turn on” to enable Linux on your Chromebook on the new screen that appears.
Settings – Linux Apps
After successfully installing and enabling Linux, you should be able to launch the Terminal app from your applications menu.
Install a Code Editor
Since you will be required to write some code/ scripts to convert your NetCDF file to CSV, you will need a text editor. One of the best editors is VS Code by Microsoft, and it supports multiple programming languages and different extensions that ease your programming/ coding process. Please look at our post that gives you a detailed guide on installing VS Code on Chromebook.
VS Code installed on Chromebook
Method 1: Python Script
The first method you can use to convert NetCDF files to a CSV file is in Python. Python is a high-level programming language that has gained popularity due to its straightforward syntax. It is one of the most utilized programming languages in Machine learning, data analysis, data science, and any other fields that concern big data.
Python comes with various modules that you can use to convert NetCDF files to CSV. They include xarray, pandas, numpy and many more. However, depending on the data in your NetCDF files, these modules alone might not be enough, and you will need to combine other modules and your programming skills.
-
Installing Python and Python-modules
Some development tools are installed automatically when you enable and install Linux on your Chromebook. That includes Python version 3, and you can verify that by executing the command below on the Terminal.
python3 --version
Python Version
To install the necessary Python modules you might require, you can use a Python package known as Pip3. To install Pip3 on your system, execute the command below.
sudo apt install python3-pip
Install Pip
After a successful install, you can install a module using the syntax below.
pip3 install <module_one> pip3 install xarray
Install Xarray
Below is a simple Python script to convert a NetCDF file to CSV. I am using the xarray module to transform the data in the NetCDF file to pandas Dataframes using the to_Dataframes() function.
Note: This script might not be the ultimate solution for your case since it flattens the different variables in the NetCDF file before saving it as a CSV file, making it lose the essence of the NetCDF format.
import xarray as xr
netcdf_File = 'file_one.nc'
xr.open_dataset(netcdf_File ).to_dataframe().to_csv('file_one.csv')
Method 2: Use Panoply Data Viewer
Panoply is a powerful data viewer Java software that runs on Windows, macOS, and Linux. It enables users to plot geo-referenced and other arrayed datasets. It is a Graphical tool and comes in handy when you don’t want to get your hands dirty on the Terminal or code.
To install Panoply on Chromebook, we will leverage the Linux support. Follow the steps below.
- Install Java by executing the command below on your Terminal.
sudo apt install openjdk-11-jdk
- Head over to the official NASA download page and download Panoply for Linux. You will see two options. The .zip file and the .tar.gz file. Choose any. For this particular post, we will download the
.zipfile. - After a successful download, move the file from your Downloads folder to your Linux Files directory, as shown in the image below.
Move the panoply zip file to Linux files.
- Extract/ Unzip the file using the unzip command as shown below.
sudo unzip PanoplyJ-5.0.0.zip
- When you change the directory (cd) to the newly extracted Panoply folder, you will notice several files, including the
Panoply.sh. That is the file we will use to run Panoply on our system. - Make the file executable by running the command below.
chmod +x panoply.sh
- Run the file with the command below.
./panoply.sh
That will launch the Panoply Graphical user interface, as shown in the image below.
Panoply
Select the NetCDF file that you wish to view and click Open. Alternatively, if you don’t see that small window, you can click on the ‘File‘ menu on the main application and select ‘Open.’ The small window will pop up, and you can choose your NetCDF file.
After working on your data, you can export it to a CSV file by clicking on the ‘File’ menu -> Export Data -> As CSV. That is illustrated in the image below.
Export to CSV
Panoply is quite a complex application and can be pretty tricky to use, especially for newbies. Feel free to look at the documentation and several examples of using Panoply on the official NASA page.
Method 3: Online Tools
If you have large NetCDF files, you can save your Chromebook from the hectic work and leverage some available online tools and sites. There are both free and paid utilities, and however, most of them vary their pricing depending on the size of the file. Some of the most most popular sites include Mygeodata and Anyconv.
Conver NetCDF Online
Tip: Nowadays, more and more companies rely on data to carry out their activities, which has led to many data privacy and data security issues. Before uploading your NetCDF files to any website online, ensure you have done thorough research about the site and verify that you can trust them with your data.
Method 4: Terminal Utilities (ncdump, cdo, ncks)
The other method you can use to convert NetCDF files to CSV is using several Terminal utilities available for Linux. Please note, these utilities require you to be well-versed with the Linux Terminal since they are used with a combination of other commands. These utilities include:
- ncdump: This tool reads the data in the NetCDF file and converts it to a CDL format (an ASCII representation of the NetCDF data). It supports various arguments, and you can use the -v (vars) option to convert the data to CSV. To install ncdump on your system, execute the command below.
sudo apt install netcdf-bin
- CDO: cdo, which stands for Climate Data Operators, is a set of tools used to process NetCDF data. To install CDO on your system, execute the command below.
sudo apt install cdo
- ncks: This utility combines various functionalities of different tools like ncdump, nccut, ncpaste, and ncert to process NetCDF data. To install
nckscommand on your system, execute the command below.sudo apt install nco
Conclusion
This post has given you four methods that you can use to convert NetCDF files to CSV files. Please note that how you process one file might differ from another since the data content is different. If you are well-versed with Python programming, writing a simple script would be a better solution. However, if you don’t want to get your hands dirty, you can utilize Panoply, which gives you an intuitive Graphical User Interface to process and plot your data. Do you have any questions regarding this topic? If yes, please feel free to hit the comments below.
