我正在尝试将数据加载到Windowsdocker桌面内置的docker容器中的clickhouse中。我在R中准备了模拟数据,用csv编写,并在Clickhouse中创建了表(我省略了连接):
library(dplyr) library(data.table) library(clickhouse) setwd("C:/Users/xxxx/Documents/testing_load") my_df = data.table(datetime = as.character(c("2018-01-01 11:21:00", "2019-01-01 11:45:00"))) c(2018, 2019) %>% lapply(function(y) { print(y) fwrite(my_df[substr(datetime,1,4) == y], paste("test_",y,".csv"), row.names = F, col.names = F ) }) dbSendQuery(con, paste( "CREATE TABLE test( datetime DateTime ) ENGINE = Log;" ) )
我试图加载的数据非常庞大,所以这只是一个示例,以显示 设置以及为什么我在第一行出现错误。我想像这样在 Powershell中使用Clickhouse客户端访问Docker容器来加载数据:
#loop through files and load $files = Get-ChildItem "C:\Users\xxxx\Documents\testing_load" foreach ($f in $files){ $outfile = $f.FullName | Write-Host Import-Csv –Delimiter "," $f.FullName | Write-Host Get-Date | Write-Host "Start loading" + $f.FullName | Write-Host docker run -it --rm --link chanalytics:clickhouse-server yandex/clickhouse-client --host clickhouse-server clickhouse-client --query="INSERT INTO test FORMAT CSV" Get-Date | Write-Host "End loading" + $f.FullName | Write-Host }
我还添加了读取数据以防万一,但是我从Clickhouse收到有关日期时间的错误消息。我试过在R中切换到positxc,这没有什么区别。我不能说,但感觉这是我不了解的非常简单的事情。以下是我得到的错误:
打开excel我可以看到datetime格式正确。任何关于什么的建议可能是怎么回事?
I was able to solve the issue:
#loop through files $files = Get-ChildItem "C:\Users\xxxx\Documents\testing_load" foreach ($f in $files){ $outfile = $f.FullName | Write-Host Get-Date | Write-Host "Start loading" + $f.FullName | Write-Host cat $f.FullName | docker run -i --rm --link chanalytics:clickhouse-client yandex/clickhouse-client -m --host chanalytics --query="INSERT INTO tpep FORMAT CSV" Get-Date | Write-Host "End loading" + $f.FullName | Write-Host }