Skip to main content

Load Data from Local File System

Using HTTP API v1/streaming_load to load data from local file into Databend. Currently, we only support CSV and Parquet file format.

Note: CSV file should be UTF-8 character encoded if you have extra character set

Before You Begin​

Step 1. Data Files for Loading​

Download the sample data file(Choose CSV or Parquet), the file contains two records:

Transaction Processing,Jim Gray,1992
Readings in Database Systems,Michael Stonebraker,2004

Download books.csv

Step 2. Create Database and Table​

mysql -h127.0.0.1 -uroot -P3307
mysql>
create database book_db;
mysql>
use book_db;
mysql>
create table books
(
    title VARCHAR(255),
    author VARCHAR(255),
    date VARCHAR(255)
);

Step 3. Load Data into the Target Tables​

Request
echo curl -H \"insert_sql:insert into book_db.books format CSV\" -H \"skip_header:0\" -H \"field_delimiter:','\" -H \"record_delimiter:'\n'\" -F  \"upload=@./books.csv\" -XPUT http://127.0.0.1:8081/v1/streaming_load|bash
Response
{
  "id": "f4c557d3-f798-4cea-960a-0ba021dd4646",
  "state": "SUCCESS",
  "stats": {
    "rows": 2,
    "bytes": 157
  },
  "error": null
}
tip

  • http://127.0.0.1:8081/v1/streaming_load

    • 127.0.0.1 is http_handler_host value in your databend-query.toml
    • 8081 is http_handler_port value in your databend-query.toml

  • skip_header: Number of lines at the start of the file to skip

  • field_delimiter: One character that separate fields

  • record_delimiter: One character that separate records

  • -F \"upload=@./books.csv\"

    • Your books.csv file location

Step 4. Verify the Loaded Data​

mysql>
select * from books;
+------------------------------+----------------------+-------+
| title                        | author               | date  |
+------------------------------+----------------------+-------+
| Transaction Processing       |  Jim Gray            |  1992 |
| Readings in Database Systems |  Michael Stonebraker |  2004 |
+------------------------------+----------------------+-------+

Step 5. Congratulations!​

You have successfully completed the tutorial.