Data is messy, 80% of you data analysis time is spent creating tidy data. Truii has a large (and growing) collection of data wrangling functionality to allow you to get your data in order and keep it in order. As you wrangle your data, you keep it in version control so that you can always go back to a previous version, or you can create new ‘child’ versions or subsets of data for analysis but keep a clear record of your data lineage.
Table data wrangling
Our most common data format is table data – a single worksheet in Microsoft excel or a CSV file. This is a table of rows and columns of data.
Our wrangling functionality for data tables includes:
- Merging: combining two data sets into one or loading additional data over an existing file.
- Filtering: creating a subset of data from a larger table.
- Fill Gaps: find empty cells and replace them
- Search and replace: Inconsistent use of certain terms.
- Remove outliers: Find values that do not match the rest of your file.
- Remove duplicate rows: remove duplicate rows and remove empty rows
Time series data wrangling
Time series data is simply table data where one of the columns has a date and time. There are a bunch of data manipulations that one routinely conducts on time series data.
- Smooth(moving average): smooth high frequency data by averaging through time.
- Manage time step: alter the date of your data by increasing or decreasing the frequency of data.Also, alter the time step of your data to make it regular.
- Correct Drift: automated loggers can drift in their calibration over time. How can you correct data once you realise the calibration is out?
Spatial data comes in a variety of formats. In our early days, Truii is working on firstly being able to visualise the commonly recognised spatial data formats like KML, SHP, and GeoJSON. Truii allows you to map whole data-sets to a location, map each column to a different location or even allow each row to be mapped to different locations that can vary through time.
See more on Truii’s spatial capabilities