[2021-03-31 18:04:55.211] [global] [error] could not find backend, defaulting to local
[2021-03-31 18:04:55.211] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-31 18:04:55.211] [local ee] [info] initializing LLVM backend
[2021-03-31 18:04:55.211] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-31 18:04:55.212] [LLVM] [info] compiling code for skylake-avx512
[2021-03-31 18:04:55.212] [memory] [info] allocated bitmap managed memory region (32.00 GB, 32.00 MB block size)
[2021-03-31 18:04:55.212] [local execution engine] [info] started driver (32.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-31 18:04:55.213] [fileinputoperator] [info] found 1 file (9.31 GB) to process.
[2021-03-31 18:04:55.213] [global] [info] sampled file:///data/zillow/Z1_preprocessed/zillow_Z1_10G.csv on 256.00 KB
[2021-03-31 18:04:55.348] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.363] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.371] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.393] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.413] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.426] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.457] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.471] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:04:55.474] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:04:55.476] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:04:55.478] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:04:55.483] [logical planner] [info] logical optimization took 0.008909ms
[2021-03-31 18:04:55.484] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.487] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.491] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.495] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.497] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.499] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.501] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:04:55.506] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:04:55.506] [global] [info] performing static typing for UDF in operator select
output type of specialized pipeline is: (str,str,str,str,str,i64,i64,i64,str,str,i64)
is this the most outer stage?: 1
[2021-03-31 18:04:55.506] [codegen] [info] generating pipeline for (str,str,str,str,f64,str,str,str) -> (str,str,str,str,str,i64,i64,i64,str,str,i64) (12 operators pipelined)
[2021-03-31 18:04:55.507] [codegen] [info] generating function extractBd for ((str,str,str,str,f64,str,str,str)) -> i64
[2021-03-31 18:04:55.508] [codegen] [info] generating lambda function for ((str,str,str,str,f64,str,str,str,i64)) -> boolean
[2021-03-31 18:04:55.508] [codegen] [info] generating function extractType for ((str,str,str,str,f64,str,str,str,i64)) -> str
[2021-03-31 18:04:55.509] [codegen] [info] generating lambda function for ((str,str,str,str,f64,str,str,str,i64,str)) -> boolean
[2021-03-31 18:04:55.509] [codegen] [info] generating lambda function for ((str,str,str,str,f64,str,str,str,i64,str)) -> str
[2021-03-31 18:04:55.510] [codegen] [info] generating lambda function for (str) -> str
[2021-03-31 18:04:55.510] [codegen] [info] generating function extractBa for ((str,str,str,str,f64,str,str,str,i64,str,str)) -> i64
[2021-03-31 18:04:55.511] [codegen] [info] generating function extractSqft for ((str,str,str,str,f64,str,str,str,i64,str,str,i64)) -> i64
[2021-03-31 18:04:55.512] [codegen] [info] generating function extractOffer for ((str,str,str,str,f64,str,str,str,i64,str,str,i64,i64)) -> str
[2021-03-31 18:04:55.512] [codegen] [info] generating function extractPrice for ((str,str,str,str,f64,str,str,str,i64,str,str,i64,i64,str)) -> i64
[2021-03-31 18:04:55.513] [codegen] [info] generating lambda function for ((str,str,str,str,f64,i64,str,str,i64,str,str,i64,i64,str)) -> boolean
[2021-03-31 18:04:55.514] [codegen] [info] generating lambda function for ((str,str,str,str,f64,i64,str,str,i64,str,str,i64,i64,str)) -> (str,str,str,str,str,i64,i64,i64,str,str,i64)
[2021-03-31 18:04:55.515] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-31 18:04:55.516] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-31 18:04:55.516] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-31 18:04:55.517] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-31 18:04:55.518] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-31 18:04:55.518] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-31 18:04:55.519] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> i64
[2021-03-31 18:04:55.519] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,i64)) -> i64
[2021-03-31 18:04:55.520] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,i64,i64)) -> str
[2021-03-31 18:04:55.521] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,i64,i64,str)) -> i64
[2021-03-31 18:04:55.522] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,i64,i64,str)) -> boolean
[2021-03-31 18:04:55.523] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,i64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,i64,i64,str,str,i64)
[2021-03-31 18:04:55.552] [global] [info] Stage0 depends on: 
[2021-03-31 18:04:55.853] [global] [info] Optimization via LLVM passes took 0.286560 ms
[2021-03-31 18:04:55.983] [global] [info] Compiled code paths for stage 0 in 0.13 ms
[2021-03-31 18:04:55.984] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.431741s
[2021-03-31 18:04:55.984] [local ee] [info] split /data/zillow/Z1_preprocessed/zillow_Z1_10G.csv into 149 tasks
[2021-03-31 18:04:56.194] [driver] [info] [Task Finished] Transform to mem in 0.210195s (5710 normal rows, 0 exceptions)
[2021-03-31 18:04:56.403] [driver] [info] [Task Finished] Transform to mem in 0.208548s (5758 normal rows, 0 exceptions)
[2021-03-31 18:04:56.611] [driver] [info] [Task Finished] Transform to mem in 0.208542s (5772 normal rows, 0 exceptions)
[2021-03-31 18:04:56.819] [driver] [info] [Task Finished] Transform to mem in 0.208205s (5744 normal rows, 0 exceptions)
[2021-03-31 18:04:57.029] [driver] [info] [Task Finished] Transform to mem in 0.209169s (5860 normal rows, 0 exceptions)
[2021-03-31 18:04:57.238] [driver] [info] [Task Finished] Transform to mem in 0.209814s (5864 normal rows, 0 exceptions)
[2021-03-31 18:04:57.447] [driver] [info] [Task Finished] Transform to mem in 0.208139s (5799 normal rows, 0 exceptions)
[2021-03-31 18:04:57.655] [driver] [info] [Task Finished] Transform to mem in 0.208068s (5663 normal rows, 0 exceptions)
[2021-03-31 18:04:57.863] [driver] [info] [Task Finished] Transform to mem in 0.208261s (5796 normal rows, 0 exceptions)
[2021-03-31 18:04:58.072] [driver] [info] [Task Finished] Transform to mem in 0.208753s (5770 normal rows, 0 exceptions)
[2021-03-31 18:04:58.280] [driver] [info] [Task Finished] Transform to mem in 0.208252s (5748 normal rows, 0 exceptions)
[2021-03-31 18:04:58.490] [driver] [info] [Task Finished] Transform to mem in 0.209551s (5779 normal rows, 0 exceptions)
[2021-03-31 18:04:58.699] [driver] [info] [Task Finished] Transform to mem in 0.209115s (5802 normal rows, 0 exceptions)
[2021-03-31 18:04:58.909] [driver] [info] [Task Finished] Transform to mem in 0.209895s (5706 normal rows, 0 exceptions)
[2021-03-31 18:04:59.118] [driver] [info] [Task Finished] Transform to mem in 0.208950s (5882 normal rows, 0 exceptions)
[2021-03-31 18:04:59.330] [driver] [info] [Task Finished] Transform to mem in 0.212048s (5883 normal rows, 0 exceptions)
[2021-03-31 18:04:59.538] [driver] [info] [Task Finished] Transform to mem in 0.208391s (6027 normal rows, 0 exceptions)
[2021-03-31 18:04:59.750] [driver] [info] [Task Finished] Transform to mem in 0.212305s (5766 normal rows, 0 exceptions)
[2021-03-31 18:04:59.958] [driver] [info] [Task Finished] Transform to mem in 0.208125s (5706 normal rows, 0 exceptions)
[2021-03-31 18:05:00.166] [driver] [info] [Task Finished] Transform to mem in 0.207853s (5709 normal rows, 0 exceptions)
[2021-03-31 18:05:00.374] [driver] [info] [Task Finished] Transform to mem in 0.208049s (5678 normal rows, 0 exceptions)
[2021-03-31 18:05:00.587] [driver] [info] [Task Finished] Transform to mem in 0.212921s (5821 normal rows, 0 exceptions)
[2021-03-31 18:05:00.796] [driver] [info] [Task Finished] Transform to mem in 0.208259s (5752 normal rows, 0 exceptions)
[2021-03-31 18:05:01.004] [driver] [info] [Task Finished] Transform to mem in 0.208118s (5761 normal rows, 0 exceptions)
[2021-03-31 18:05:01.212] [driver] [info] [Task Finished] Transform to mem in 0.208559s (5882 normal rows, 0 exceptions)
[2021-03-31 18:05:01.421] [driver] [info] [Task Finished] Transform to mem in 0.208229s (5651 normal rows, 0 exceptions)
[2021-03-31 18:05:01.629] [driver] [info] [Task Finished] Transform to mem in 0.208564s (5772 normal rows, 0 exceptions)
[2021-03-31 18:05:01.837] [driver] [info] [Task Finished] Transform to mem in 0.208001s (5665 normal rows, 0 exceptions)
[2021-03-31 18:05:02.046] [driver] [info] [Task Finished] Transform to mem in 0.208648s (5716 normal rows, 0 exceptions)
[2021-03-31 18:05:02.255] [driver] [info] [Task Finished] Transform to mem in 0.208794s (5723 normal rows, 0 exceptions)
[2021-03-31 18:05:02.463] [driver] [info] [Task Finished] Transform to mem in 0.208669s (5792 normal rows, 0 exceptions)
[2021-03-31 18:05:02.672] [driver] [info] [Task Finished] Transform to mem in 0.208622s (5781 normal rows, 0 exceptions)
[2021-03-31 18:05:02.880] [driver] [info] [Task Finished] Transform to mem in 0.208141s (5622 normal rows, 0 exceptions)
[2021-03-31 18:05:03.092] [driver] [info] [Task Finished] Transform to mem in 0.211557s (5781 normal rows, 0 exceptions)
[2021-03-31 18:05:03.302] [driver] [info] [Task Finished] Transform to mem in 0.210157s (5842 normal rows, 0 exceptions)
[2021-03-31 18:05:03.512] [driver] [info] [Task Finished] Transform to mem in 0.209733s (5926 normal rows, 0 exceptions)
[2021-03-31 18:05:03.720] [driver] [info] [Task Finished] Transform to mem in 0.208583s (5860 normal rows, 0 exceptions)
[2021-03-31 18:05:03.930] [driver] [info] [Task Finished] Transform to mem in 0.209739s (5749 normal rows, 0 exceptions)
[2021-03-31 18:05:04.138] [driver] [info] [Task Finished] Transform to mem in 0.208328s (5696 normal rows, 0 exceptions)
[2021-03-31 18:05:04.347] [driver] [info] [Task Finished] Transform to mem in 0.208651s (5821 normal rows, 0 exceptions)
[2021-03-31 18:05:04.555] [driver] [info] [Task Finished] Transform to mem in 0.208475s (5831 normal rows, 0 exceptions)
[2021-03-31 18:05:04.764] [driver] [info] [Task Finished] Transform to mem in 0.208668s (5915 normal rows, 0 exceptions)
[2021-03-31 18:05:04.974] [driver] [info] [Task Finished] Transform to mem in 0.209953s (5721 normal rows, 0 exceptions)
[2021-03-31 18:05:05.184] [driver] [info] [Task Finished] Transform to mem in 0.210059s (5839 normal rows, 0 exceptions)
[2021-03-31 18:05:05.394] [driver] [info] [Task Finished] Transform to mem in 0.209797s (5750 normal rows, 0 exceptions)
[2021-03-31 18:05:05.604] [driver] [info] [Task Finished] Transform to mem in 0.209753s (5878 normal rows, 0 exceptions)
[2021-03-31 18:05:05.813] [driver] [info] [Task Finished] Transform to mem in 0.209212s (5861 normal rows, 0 exceptions)
[2021-03-31 18:05:06.022] [driver] [info] [Task Finished] Transform to mem in 0.208545s (5810 normal rows, 0 exceptions)
[2021-03-31 18:05:06.230] [driver] [info] [Task Finished] Transform to mem in 0.208451s (5662 normal rows, 0 exceptions)
[2021-03-31 18:05:06.439] [driver] [info] [Task Finished] Transform to mem in 0.209099s (5777 normal rows, 0 exceptions)
[2021-03-31 18:05:06.650] [driver] [info] [Task Finished] Transform to mem in 0.210580s (5657 normal rows, 0 exceptions)
[2021-03-31 18:05:06.861] [driver] [info] [Task Finished] Transform to mem in 0.211098s (5731 normal rows, 0 exceptions)
[2021-03-31 18:05:07.073] [driver] [info] [Task Finished] Transform to mem in 0.211803s (5694 normal rows, 0 exceptions)
[2021-03-31 18:05:07.282] [driver] [info] [Task Finished] Transform to mem in 0.209255s (5813 normal rows, 0 exceptions)
[2021-03-31 18:05:07.492] [driver] [info] [Task Finished] Transform to mem in 0.209761s (5893 normal rows, 0 exceptions)
[2021-03-31 18:05:07.701] [driver] [info] [Task Finished] Transform to mem in 0.209576s (5850 normal rows, 0 exceptions)
[2021-03-31 18:05:07.911] [driver] [info] [Task Finished] Transform to mem in 0.209259s (5742 normal rows, 0 exceptions)
[2021-03-31 18:05:08.120] [driver] [info] [Task Finished] Transform to mem in 0.209760s (5795 normal rows, 0 exceptions)
[2021-03-31 18:05:08.329] [driver] [info] [Task Finished] Transform to mem in 0.208154s (5714 normal rows, 0 exceptions)
[2021-03-31 18:05:08.541] [driver] [info] [Task Finished] Transform to mem in 0.212601s (5709 normal rows, 0 exceptions)
[2021-03-31 18:05:08.750] [driver] [info] [Task Finished] Transform to mem in 0.208669s (5750 normal rows, 0 exceptions)
[2021-03-31 18:05:08.958] [driver] [info] [Task Finished] Transform to mem in 0.208647s (5750 normal rows, 0 exceptions)
[2021-03-31 18:05:09.167] [driver] [info] [Task Finished] Transform to mem in 0.208596s (5759 normal rows, 0 exceptions)
[2021-03-31 18:05:09.376] [driver] [info] [Task Finished] Transform to mem in 0.208478s (5734 normal rows, 0 exceptions)
[2021-03-31 18:05:09.585] [driver] [info] [Task Finished] Transform to mem in 0.209557s (5881 normal rows, 0 exceptions)
[2021-03-31 18:05:09.793] [driver] [info] [Task Finished] Transform to mem in 0.208084s (5807 normal rows, 0 exceptions)
[2021-03-31 18:05:10.003] [driver] [info] [Task Finished] Transform to mem in 0.209395s (5797 normal rows, 0 exceptions)
[2021-03-31 18:05:10.211] [driver] [info] [Task Finished] Transform to mem in 0.208520s (5786 normal rows, 0 exceptions)
[2021-03-31 18:05:10.423] [driver] [info] [Task Finished] Transform to mem in 0.211994s (5826 normal rows, 0 exceptions)
[2021-03-31 18:05:10.632] [driver] [info] [Task Finished] Transform to mem in 0.209057s (5722 normal rows, 0 exceptions)
[2021-03-31 18:05:10.845] [driver] [info] [Task Finished] Transform to mem in 0.212452s (5746 normal rows, 0 exceptions)
[2021-03-31 18:05:11.053] [driver] [info] [Task Finished] Transform to mem in 0.207861s (5775 normal rows, 0 exceptions)
[2021-03-31 18:05:11.261] [driver] [info] [Task Finished] Transform to mem in 0.208264s (5766 normal rows, 0 exceptions)
[2021-03-31 18:05:11.470] [driver] [info] [Task Finished] Transform to mem in 0.208768s (5877 normal rows, 0 exceptions)
[2021-03-31 18:05:11.680] [driver] [info] [Task Finished] Transform to mem in 0.210300s (5677 normal rows, 0 exceptions)
[2021-03-31 18:05:11.889] [driver] [info] [Task Finished] Transform to mem in 0.208932s (5846 normal rows, 0 exceptions)
[2021-03-31 18:05:12.098] [driver] [info] [Task Finished] Transform to mem in 0.208694s (5851 normal rows, 0 exceptions)
[2021-03-31 18:05:12.306] [driver] [info] [Task Finished] Transform to mem in 0.208622s (5820 normal rows, 0 exceptions)
[2021-03-31 18:05:12.514] [driver] [info] [Task Finished] Transform to mem in 0.208099s (5726 normal rows, 0 exceptions)
[2021-03-31 18:05:12.723] [driver] [info] [Task Finished] Transform to mem in 0.208476s (5684 normal rows, 0 exceptions)
[2021-03-31 18:05:12.931] [driver] [info] [Task Finished] Transform to mem in 0.208430s (5744 normal rows, 0 exceptions)
[2021-03-31 18:05:13.141] [driver] [info] [Task Finished] Transform to mem in 0.209658s (5606 normal rows, 0 exceptions)
[2021-03-31 18:05:13.349] [driver] [info] [Task Finished] Transform to mem in 0.208409s (5707 normal rows, 0 exceptions)
[2021-03-31 18:05:13.558] [driver] [info] [Task Finished] Transform to mem in 0.208400s (5831 normal rows, 0 exceptions)
[2021-03-31 18:05:13.766] [driver] [info] [Task Finished] Transform to mem in 0.208348s (5742 normal rows, 0 exceptions)
[2021-03-31 18:05:13.974] [driver] [info] [Task Finished] Transform to mem in 0.207867s (5546 normal rows, 0 exceptions)
[2021-03-31 18:05:14.186] [driver] [info] [Task Finished] Transform to mem in 0.211874s (5744 normal rows, 0 exceptions)
[2021-03-31 18:05:14.398] [driver] [info] [Task Finished] Transform to mem in 0.211737s (5879 normal rows, 0 exceptions)
[2021-03-31 18:05:14.606] [driver] [info] [Task Finished] Transform to mem in 0.208480s (5865 normal rows, 0 exceptions)
[2021-03-31 18:05:14.815] [driver] [info] [Task Finished] Transform to mem in 0.208340s (5776 normal rows, 0 exceptions)
[2021-03-31 18:05:15.022] [driver] [info] [Task Finished] Transform to mem in 0.207787s (5632 normal rows, 0 exceptions)
[2021-03-31 18:05:15.231] [driver] [info] [Task Finished] Transform to mem in 0.208998s (5766 normal rows, 0 exceptions)
[2021-03-31 18:05:15.440] [driver] [info] [Task Finished] Transform to mem in 0.209065s (5851 normal rows, 0 exceptions)
[2021-03-31 18:05:15.650] [driver] [info] [Task Finished] Transform to mem in 0.209688s (5842 normal rows, 0 exceptions)
[2021-03-31 18:05:15.860] [driver] [info] [Task Finished] Transform to mem in 0.209677s (5844 normal rows, 0 exceptions)
[2021-03-31 18:05:16.069] [driver] [info] [Task Finished] Transform to mem in 0.208764s (5911 normal rows, 0 exceptions)
[2021-03-31 18:05:16.278] [driver] [info] [Task Finished] Transform to mem in 0.209319s (5907 normal rows, 0 exceptions)
[2021-03-31 18:05:16.491] [driver] [info] [Task Finished] Transform to mem in 0.213000s (5866 normal rows, 0 exceptions)
[2021-03-31 18:05:16.700] [driver] [info] [Task Finished] Transform to mem in 0.208505s (5743 normal rows, 0 exceptions)
[2021-03-31 18:05:16.908] [driver] [info] [Task Finished] Transform to mem in 0.208782s (5720 normal rows, 0 exceptions)
[2021-03-31 18:05:17.117] [driver] [info] [Task Finished] Transform to mem in 0.208322s (5685 normal rows, 0 exceptions)
[2021-03-31 18:05:17.326] [driver] [info] [Task Finished] Transform to mem in 0.208993s (5883 normal rows, 0 exceptions)
[2021-03-31 18:05:17.534] [driver] [info] [Task Finished] Transform to mem in 0.208049s (5706 normal rows, 0 exceptions)
[2021-03-31 18:05:17.746] [driver] [info] [Task Finished] Transform to mem in 0.212463s (5929 normal rows, 0 exceptions)
[2021-03-31 18:05:17.956] [driver] [info] [Task Finished] Transform to mem in 0.209355s (5872 normal rows, 0 exceptions)
[2021-03-31 18:05:18.167] [driver] [info] [Task Finished] Transform to mem in 0.211798s (5847 normal rows, 0 exceptions)
[2021-03-31 18:05:18.375] [driver] [info] [Task Finished] Transform to mem in 0.208001s (5816 normal rows, 0 exceptions)
[2021-03-31 18:05:18.584] [driver] [info] [Task Finished] Transform to mem in 0.208228s (5649 normal rows, 0 exceptions)
[2021-03-31 18:05:18.793] [driver] [info] [Task Finished] Transform to mem in 0.209098s (5873 normal rows, 0 exceptions)
[2021-03-31 18:05:19.001] [driver] [info] [Task Finished] Transform to mem in 0.208376s (5686 normal rows, 0 exceptions)
[2021-03-31 18:05:19.210] [driver] [info] [Task Finished] Transform to mem in 0.208443s (5806 normal rows, 0 exceptions)
[2021-03-31 18:05:19.419] [driver] [info] [Task Finished] Transform to mem in 0.209309s (5765 normal rows, 0 exceptions)
[2021-03-31 18:05:19.627] [driver] [info] [Task Finished] Transform to mem in 0.208355s (5770 normal rows, 0 exceptions)
[2021-03-31 18:05:19.836] [driver] [info] [Task Finished] Transform to mem in 0.208552s (5643 normal rows, 0 exceptions)
[2021-03-31 18:05:20.044] [driver] [info] [Task Finished] Transform to mem in 0.208257s (5951 normal rows, 0 exceptions)
[2021-03-31 18:05:20.253] [driver] [info] [Task Finished] Transform to mem in 0.208544s (5759 normal rows, 0 exceptions)
[2021-03-31 18:05:20.460] [driver] [info] [Task Finished] Transform to mem in 0.207646s (5698 normal rows, 0 exceptions)
[2021-03-31 18:05:20.670] [driver] [info] [Task Finished] Transform to mem in 0.209927s (5955 normal rows, 0 exceptions)
[2021-03-31 18:05:20.879] [driver] [info] [Task Finished] Transform to mem in 0.208833s (5768 normal rows, 0 exceptions)
[2021-03-31 18:05:21.088] [driver] [info] [Task Finished] Transform to mem in 0.209067s (5772 normal rows, 0 exceptions)
[2021-03-31 18:05:21.297] [driver] [info] [Task Finished] Transform to mem in 0.208321s (5802 normal rows, 0 exceptions)
[2021-03-31 18:05:21.508] [driver] [info] [Task Finished] Transform to mem in 0.211468s (5799 normal rows, 0 exceptions)
[2021-03-31 18:05:21.720] [driver] [info] [Task Finished] Transform to mem in 0.212056s (5862 normal rows, 0 exceptions)
[2021-03-31 18:05:21.929] [driver] [info] [Task Finished] Transform to mem in 0.208664s (5856 normal rows, 0 exceptions)
[2021-03-31 18:05:22.137] [driver] [info] [Task Finished] Transform to mem in 0.208353s (5685 normal rows, 0 exceptions)
[2021-03-31 18:05:22.346] [driver] [info] [Task Finished] Transform to mem in 0.208473s (5751 normal rows, 0 exceptions)
[2021-03-31 18:05:22.555] [driver] [info] [Task Finished] Transform to mem in 0.208994s (5891 normal rows, 0 exceptions)
[2021-03-31 18:05:22.763] [driver] [info] [Task Finished] Transform to mem in 0.208746s (5852 normal rows, 0 exceptions)
[2021-03-31 18:05:22.973] [driver] [info] [Task Finished] Transform to mem in 0.209114s (5722 normal rows, 0 exceptions)
[2021-03-31 18:05:23.181] [driver] [info] [Task Finished] Transform to mem in 0.208175s (5645 normal rows, 0 exceptions)
[2021-03-31 18:05:23.389] [driver] [info] [Task Finished] Transform to mem in 0.207883s (5788 normal rows, 0 exceptions)
[2021-03-31 18:05:23.597] [driver] [info] [Task Finished] Transform to mem in 0.208604s (5813 normal rows, 0 exceptions)
[2021-03-31 18:05:23.806] [driver] [info] [Task Finished] Transform to mem in 0.208342s (5674 normal rows, 0 exceptions)
[2021-03-31 18:05:24.014] [driver] [info] [Task Finished] Transform to mem in 0.208254s (5853 normal rows, 0 exceptions)
[2021-03-31 18:05:24.223] [driver] [info] [Task Finished] Transform to mem in 0.209285s (5848 normal rows, 0 exceptions)
[2021-03-31 18:05:24.433] [driver] [info] [Task Finished] Transform to mem in 0.210086s (5811 normal rows, 0 exceptions)
[2021-03-31 18:05:24.646] [driver] [info] [Task Finished] Transform to mem in 0.212981s (5732 normal rows, 0 exceptions)
[2021-03-31 18:05:24.855] [driver] [info] [Task Finished] Transform to mem in 0.208719s (5801 normal rows, 0 exceptions)
[2021-03-31 18:05:25.068] [driver] [info] [Task Finished] Transform to mem in 0.212966s (5827 normal rows, 0 exceptions)
[2021-03-31 18:05:25.276] [driver] [info] [Task Finished] Transform to mem in 0.208398s (5868 normal rows, 0 exceptions)
[2021-03-31 18:05:25.489] [driver] [info] [Task Finished] Transform to mem in 0.212464s (5794 normal rows, 0 exceptions)
[2021-03-31 18:05:25.698] [driver] [info] [Task Finished] Transform to mem in 0.209407s (5817 normal rows, 0 exceptions)
[2021-03-31 18:05:25.907] [driver] [info] [Task Finished] Transform to mem in 0.208491s (5696 normal rows, 0 exceptions)
[2021-03-31 18:05:26.115] [driver] [info] [Task Finished] Transform to mem in 0.208488s (5732 normal rows, 0 exceptions)
[2021-03-31 18:05:26.324] [driver] [info] [Task Finished] Transform to mem in 0.209014s (5629 normal rows, 0 exceptions)
[2021-03-31 18:05:26.533] [driver] [info] [Task Finished] Transform to mem in 0.208614s (5789 normal rows, 0 exceptions)
[2021-03-31 18:05:26.742] [driver] [info] [Task Finished] Transform to mem in 0.208530s (5740 normal rows, 0 exceptions)
[2021-03-31 18:05:26.950] [driver] [info] [Task Finished] Transform to mem in 0.208666s (5841 normal rows, 0 exceptions)
[2021-03-31 18:05:27.161] [driver] [info] [Task Finished] Transform to mem in 0.210877s (5857 normal rows, 0 exceptions)
[2021-03-31 18:05:27.161] [global] [info] [Transform Stage] Stage 0 completed 149 load&transform tasks in 31.1775s
[2021-03-31 18:05:27.161] [global] [info] [Transform Stage] Stage 0 total wall clock time: 31.175s, 48740298 input rows, time to process 1 row via fast path: 0.000639614ms

[2021-03-31 18:05:27.175] [global] [info] Writing 861387 rows as output to file (116.34 MB)
[2021-03-31 18:05:27.304] [driver] [info] [Task Finished] write to file in 0.128628s (861387 rows, 116.34 MB)
[2021-03-31 18:05:27.304] [global] [info] writing output took 0.142443s
[2021-03-31 18:05:27.304] [global] [info] [Transform Stage] Stage 0 completed 149 sink tasks in 0.142523s
[2021-03-31 18:05:27.304] [global] [info] [Transform Stage] Stage 0 took 31.7518s
[2021-03-31 18:05:27.304] [global] [info] Query Execution took 31.8321s. (planning: 0.0801991s, execution: 31.7519s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 31 2021, 12:55:20) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z1_preprocessed/zillow_Z1_10G.csv']
Tuplex startup time: 0.004955768585205078
Tuplex job time: 32.09250259399414 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': True, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 0, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '32G', 'tuplex.env.hostname': '499bbd619a75', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '8MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.00890887,"llvm_optimization_time_s":0.28656,"llvm_compilation_time_s":0.130613,"total_compilation_time_s":0.431739,"sampling_time_s":0.111024,"stages":[{"stage_no":0,"fast_path_wall_time_s":31.175,"fast_path_time_s":31.1775,"fast_path_per_row_time_ns":639.614,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.004955768585205078, "jobTime": 32.09250259399414}
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
