Commit 16d56867 authored by Ian Dennis Miller's avatar Ian Dennis Miller

longer readme

parent a05d5e52
Pipeline #7 canceled with stages
...@@ -10,6 +10,10 @@ CSV Pipes can be used to build pipelines for streaming computations. ...@@ -10,6 +10,10 @@ CSV Pipes can be used to build pipelines for streaming computations.
Use the UNIX pipe "|" to chain together commands and filters. Use the UNIX pipe "|" to chain together commands and filters.
The syntax is reminiscent of SQL. The syntax is reminiscent of SQL.
CSV Pipes is still under development.
It is likely that some CSV files will fail to parse correctly.
In that case, please open an Issue about the problem.
## Installation ## Installation
``` ```
...@@ -18,16 +22,31 @@ source csv-pipes.sh ...@@ -18,16 +22,31 @@ source csv-pipes.sh
csv.install csv.install
``` ```
## Examples As an alternative to the automatic procedure:
1. download the file `csv-pipes.sh` ([here](https://projects.sisrlab.com/idm/csv-pipes/raw/master/csv-pipes.sh)) and place it in your home directory
2. Add the following line at the end of your `.bashrc` or `.zshrc` file:
`source $HOME/csv-pipes.sh`
Now CSV Pipes is installed.
## Usage
The following examples provide a brief tour of how to use CSV Pipes.
CSV Pipes provides the `csv` shell function.
Once CSV Pipes has been activated, many `csv` subcommands become available.
### Display available commands ### Display available commands
We start by getting a list of all the available commands with `csv help`.
``` ```
bash$ csv help bash$ csv help
CSV Pipes 0.1 CSV Pipes 0.1
Usage: Usage:
csv \[command\] \[options\] csv [command] [options]
Available commands: Available commands:
read [file] Read a CSV file. May be gzipped. read [file] Read a CSV file. May be gzipped.
...@@ -50,6 +69,9 @@ https://projects.sisrlab.com/idm/csv-pipes ...@@ -50,6 +69,9 @@ https://projects.sisrlab.com/idm/csv-pipes
### Obtain file listing as CSV ### Obtain file listing as CSV
We will gather some basic CSV data so we can go through several examples.
The `csv ls` function will render the current directory as CSV format.
``` ```
bash$ csv ls bash$ csv ls
-rw-rw-r--,1,idm,idm,3999,Oct,1,02:53,csv-pipes.sh -rw-rw-r--,1,idm,idm,3999,Oct,1,02:53,csv-pipes.sh
...@@ -58,6 +80,9 @@ bash$ csv ls ...@@ -58,6 +80,9 @@ bash$ csv ls
### Obtain file listing as CSV and select file sizes ### Obtain file listing as CSV and select file sizes
Use `csv select` to get a single column from the results.
The results of `csv ls` are piped directly to `csv select`.
``` ```
bash$ csv ls | csv select 5 bash$ csv ls | csv select 5
3999 3999
...@@ -66,6 +91,8 @@ bash$ csv ls | csv select 5 ...@@ -66,6 +91,8 @@ bash$ csv ls | csv select 5
### Calculate average file size ### Calculate average file size
Perform a simple calculation with the results.
``` ```
bash$ csv ls | csv select 5 | csv mean bash$ csv ls | csv select 5 | csv mean
1915.66666667 1915.66666667
...@@ -73,6 +100,9 @@ bash$ csv ls | csv select 5 | csv mean ...@@ -73,6 +100,9 @@ bash$ csv ls | csv select 5 | csv mean
### Limit results to 5 ### Limit results to 5
Use `csv limit` to stop gathering results once the desired quantity has been reached.
This has the effect of stopping upstream computations.
``` ```
bash$ csv ls / | csv limit 5 bash$ csv ls / | csv limit 5
drwxr-xr-x,2,root,root,4096,Sep,12,06:40,bin drwxr-xr-x,2,root,root,4096,Sep,12,06:40,bin
...@@ -84,38 +114,26 @@ drwxr-xr-x,5,root,root,4096,Aug,7,20:53,home ...@@ -84,38 +114,26 @@ drwxr-xr-x,5,root,root,4096,Aug,7,20:53,home
### Write CSV to file ### Write CSV to file
In order to store CSV Pipes results in a file, use `>` - the UNIX file redirect symbol.
``` ```
bash$ csv ls / > /tmp/listing.csv bash$ csv ls / > /tmp/listing.csv
``` ```
### Read CSV from a file ### Read CSV from a file
CSV results may be read from a file using `csv read`, which is an enhanced version of the UNIX redirect `<`.
``` ```
bash$ csv read /tmp/listing.csv | csv limit 2 bash$ csv read /tmp/listing.csv | csv limit 2
drwxr-xr-x,2,root,root,4096,Sep,12,06:40,bin drwxr-xr-x,2,root,root,4096,Sep,12,06:40,bin
drwxr-xr-x,4,root,root,4096,Sep,13,06:35,boot drwxr-xr-x,4,root,root,4096,Sep,13,06:35,boot
``` ```
## Commands ## About
- read [file]
- select [cols]
- names [file]
- limit [num]
- ls [path]
- benchmark [file]
- version
- help
## Filters
- collapse (piped) Ian Dennis Miller
- unquote (piped)
- split (piped)
- sum (piped)
- mean (piped)
- count (piped)
## About http://imiller.utsc.utoronto.ca
https://www.sisrlab.com https://www.sisrlab.com
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment