[2021-03-23 12:33:54.764] [global] [error] could not find backend, defaulting to local
[2021-03-23 12:33:54.765] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-23 12:33:54.765] [local ee] [info] initializing LLVM backend
[2021-03-23 12:33:54.765] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-23 12:33:54.765] [LLVM] [info] compiling code for skylake-avx512
[2021-03-23 12:33:54.766] [memory] [info] allocated bitmap managed memory region (32.00 GB, 32.00 MB block size)
[2021-03-23 12:33:54.766] [local execution engine] [info] started driver (32.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-23 12:33:54.766] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2021-03-23 12:33:54.766] [global] [info] sampled file:///data/zillow/Z2_preprocessed/zillow_Z2_10G.csv on 256.00 KB
[2021-03-23 12:33:54.900] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:33:54.916] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:33:54.924] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:33:54.950] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:33:54.970] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:33:54.983] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:33:55.014] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-23 12:33:55.031] [global] [info] performing static typing for UDF in operator map
[2021-03-23 12:33:55.033] [global] [info] performing static typing for UDF in operator map
[2021-03-23 12:33:55.035] [global] [info] performing static typing for UDF in operator map
[2021-03-23 12:33:55.038] [global] [info] performing static typing for UDF in operator map
[2021-03-23 12:33:55.053] [logical planner] [info] logical optimization took 0.019926ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-23 12:33:55.054] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (13 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-23 12:33:55.054] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-23 12:33:55.055] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-23 12:33:55.055] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-23 12:33:55.056] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-23 12:33:55.056] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-23 12:33:55.057] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-23 12:33:55.058] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> f64
[2021-03-23 12:33:55.059] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64)) -> i64
[2021-03-23 12:33:55.060] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64)) -> str
[2021-03-23 12:33:55.060] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64,str)) -> i64
[2021-03-23 12:33:55.062] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-23 12:33:55.062] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-23 12:33:55.063] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-23 12:33:55.089] [global] [info] Stage0 depends on: 
[2021-03-23 12:33:55.234] [global] [info] Optimization via LLVM passes took 0.138056 ms
[2021-03-23 12:33:55.296] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2021-03-23 12:33:55.296] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.207184s
[2021-03-23 12:33:55.296] [local ee] [info] split /data/zillow/Z2_preprocessed/zillow_Z2_10G.csv into 160 tasks
[2021-03-23 12:33:55.557] [driver] [info] [Task Finished] Transform to mem in 0.261193s (14226 normal rows, 0 exceptions)
[2021-03-23 12:33:55.816] [driver] [info] [Task Finished] Transform to mem in 0.258497s (14206 normal rows, 0 exceptions)
[2021-03-23 12:33:56.075] [driver] [info] [Task Finished] Transform to mem in 0.258566s (13970 normal rows, 0 exceptions)
[2021-03-23 12:33:56.334] [driver] [info] [Task Finished] Transform to mem in 0.259651s (14253 normal rows, 0 exceptions)
[2021-03-23 12:33:56.593] [driver] [info] [Task Finished] Transform to mem in 0.258621s (14217 normal rows, 0 exceptions)
[2021-03-23 12:33:56.850] [driver] [info] [Task Finished] Transform to mem in 0.257279s (13920 normal rows, 0 exceptions)
[2021-03-23 12:33:57.109] [driver] [info] [Task Finished] Transform to mem in 0.258625s (14267 normal rows, 0 exceptions)
[2021-03-23 12:33:57.367] [driver] [info] [Task Finished] Transform to mem in 0.257879s (14248 normal rows, 0 exceptions)
[2021-03-23 12:33:57.624] [driver] [info] [Task Finished] Transform to mem in 0.257141s (13953 normal rows, 0 exceptions)
[2021-03-23 12:33:57.882] [driver] [info] [Task Finished] Transform to mem in 0.257780s (14207 normal rows, 0 exceptions)
[2021-03-23 12:33:58.140] [driver] [info] [Task Finished] Transform to mem in 0.258080s (14237 normal rows, 0 exceptions)
[2021-03-23 12:33:58.397] [driver] [info] [Task Finished] Transform to mem in 0.257565s (13988 normal rows, 0 exceptions)
[2021-03-23 12:33:58.656] [driver] [info] [Task Finished] Transform to mem in 0.258279s (14177 normal rows, 0 exceptions)
[2021-03-23 12:33:58.914] [driver] [info] [Task Finished] Transform to mem in 0.258120s (14242 normal rows, 0 exceptions)
[2021-03-23 12:33:59.172] [driver] [info] [Task Finished] Transform to mem in 0.258525s (14020 normal rows, 0 exceptions)
[2021-03-23 12:33:59.430] [driver] [info] [Task Finished] Transform to mem in 0.258199s (14162 normal rows, 0 exceptions)
[2021-03-23 12:33:59.689] [driver] [info] [Task Finished] Transform to mem in 0.258431s (14241 normal rows, 0 exceptions)
[2021-03-23 12:33:59.947] [driver] [info] [Task Finished] Transform to mem in 0.258035s (14059 normal rows, 0 exceptions)
[2021-03-23 12:34:00.205] [driver] [info] [Task Finished] Transform to mem in 0.257860s (14114 normal rows, 0 exceptions)
[2021-03-23 12:34:00.462] [driver] [info] [Task Finished] Transform to mem in 0.257620s (14238 normal rows, 0 exceptions)
[2021-03-23 12:34:00.720] [driver] [info] [Task Finished] Transform to mem in 0.257286s (14081 normal rows, 0 exceptions)
[2021-03-23 12:34:00.978] [driver] [info] [Task Finished] Transform to mem in 0.257850s (14095 normal rows, 0 exceptions)
[2021-03-23 12:34:01.236] [driver] [info] [Task Finished] Transform to mem in 0.258349s (14221 normal rows, 0 exceptions)
[2021-03-23 12:34:01.494] [driver] [info] [Task Finished] Transform to mem in 0.257854s (14086 normal rows, 0 exceptions)
[2021-03-23 12:34:01.752] [driver] [info] [Task Finished] Transform to mem in 0.257641s (14097 normal rows, 0 exceptions)
[2021-03-23 12:34:02.009] [driver] [info] [Task Finished] Transform to mem in 0.257529s (14212 normal rows, 0 exceptions)
[2021-03-23 12:34:02.267] [driver] [info] [Task Finished] Transform to mem in 0.257488s (14108 normal rows, 0 exceptions)
[2021-03-23 12:34:02.525] [driver] [info] [Task Finished] Transform to mem in 0.257950s (14100 normal rows, 0 exceptions)
[2021-03-23 12:34:02.783] [driver] [info] [Task Finished] Transform to mem in 0.258383s (14212 normal rows, 0 exceptions)
[2021-03-23 12:34:03.041] [driver] [info] [Task Finished] Transform to mem in 0.258158s (14099 normal rows, 0 exceptions)
[2021-03-23 12:34:03.299] [driver] [info] [Task Finished] Transform to mem in 0.258145s (14094 normal rows, 0 exceptions)
[2021-03-23 12:34:03.558] [driver] [info] [Task Finished] Transform to mem in 0.258248s (14223 normal rows, 0 exceptions)
[2021-03-23 12:34:03.815] [driver] [info] [Task Finished] Transform to mem in 0.257247s (14089 normal rows, 0 exceptions)
[2021-03-23 12:34:04.072] [driver] [info] [Task Finished] Transform to mem in 0.257259s (14083 normal rows, 0 exceptions)
[2021-03-23 12:34:04.330] [driver] [info] [Task Finished] Transform to mem in 0.258187s (14235 normal rows, 0 exceptions)
[2021-03-23 12:34:04.588] [driver] [info] [Task Finished] Transform to mem in 0.257954s (14170 normal rows, 0 exceptions)
[2021-03-23 12:34:04.846] [driver] [info] [Task Finished] Transform to mem in 0.257726s (14002 normal rows, 0 exceptions)
[2021-03-23 12:34:05.104] [driver] [info] [Task Finished] Transform to mem in 0.258397s (14226 normal rows, 0 exceptions)
[2021-03-23 12:34:05.362] [driver] [info] [Task Finished] Transform to mem in 0.257326s (14206 normal rows, 0 exceptions)
[2021-03-23 12:34:05.619] [driver] [info] [Task Finished] Transform to mem in 0.257195s (13961 normal rows, 0 exceptions)
[2021-03-23 12:34:05.877] [driver] [info] [Task Finished] Transform to mem in 0.258266s (14246 normal rows, 0 exceptions)
[2021-03-23 12:34:06.136] [driver] [info] [Task Finished] Transform to mem in 0.258307s (14233 normal rows, 0 exceptions)
[2021-03-23 12:34:06.393] [driver] [info] [Task Finished] Transform to mem in 0.257539s (13915 normal rows, 0 exceptions)
[2021-03-23 12:34:06.652] [driver] [info] [Task Finished] Transform to mem in 0.258587s (14253 normal rows, 0 exceptions)
[2021-03-23 12:34:06.910] [driver] [info] [Task Finished] Transform to mem in 0.258254s (14267 normal rows, 0 exceptions)
[2021-03-23 12:34:07.168] [driver] [info] [Task Finished] Transform to mem in 0.258075s (13892 normal rows, 0 exceptions)
[2021-03-23 12:34:07.427] [driver] [info] [Task Finished] Transform to mem in 0.258848s (14245 normal rows, 0 exceptions)
[2021-03-23 12:34:07.685] [driver] [info] [Task Finished] Transform to mem in 0.258048s (14260 normal rows, 0 exceptions)
[2021-03-23 12:34:07.942] [driver] [info] [Task Finished] Transform to mem in 0.257297s (13978 normal rows, 0 exceptions)
[2021-03-23 12:34:08.200] [driver] [info] [Task Finished] Transform to mem in 0.258035s (14181 normal rows, 0 exceptions)
[2021-03-23 12:34:08.459] [driver] [info] [Task Finished] Transform to mem in 0.258570s (14234 normal rows, 0 exceptions)
[2021-03-23 12:34:08.717] [driver] [info] [Task Finished] Transform to mem in 0.258149s (14015 normal rows, 0 exceptions)
[2021-03-23 12:34:08.976] [driver] [info] [Task Finished] Transform to mem in 0.258391s (14142 normal rows, 0 exceptions)
[2021-03-23 12:34:09.234] [driver] [info] [Task Finished] Transform to mem in 0.258050s (14280 normal rows, 0 exceptions)
[2021-03-23 12:34:09.491] [driver] [info] [Task Finished] Transform to mem in 0.257186s (13982 normal rows, 0 exceptions)
[2021-03-23 12:34:09.749] [driver] [info] [Task Finished] Transform to mem in 0.258122s (14173 normal rows, 0 exceptions)
[2021-03-23 12:34:10.008] [driver] [info] [Task Finished] Transform to mem in 0.258540s (14235 normal rows, 0 exceptions)
[2021-03-23 12:34:10.265] [driver] [info] [Task Finished] Transform to mem in 0.257209s (14054 normal rows, 0 exceptions)
[2021-03-23 12:34:10.523] [driver] [info] [Task Finished] Transform to mem in 0.257789s (14129 normal rows, 0 exceptions)
[2021-03-23 12:34:10.780] [driver] [info] [Task Finished] Transform to mem in 0.257426s (14223 normal rows, 0 exceptions)
[2021-03-23 12:34:11.037] [driver] [info] [Task Finished] Transform to mem in 0.257018s (14081 normal rows, 0 exceptions)
[2021-03-23 12:34:11.294] [driver] [info] [Task Finished] Transform to mem in 0.257404s (14108 normal rows, 0 exceptions)
[2021-03-23 12:34:11.553] [driver] [info] [Task Finished] Transform to mem in 0.258341s (14208 normal rows, 0 exceptions)
[2021-03-23 12:34:11.811] [driver] [info] [Task Finished] Transform to mem in 0.258002s (14089 normal rows, 0 exceptions)
[2021-03-23 12:34:12.069] [driver] [info] [Task Finished] Transform to mem in 0.258103s (14094 normal rows, 0 exceptions)
[2021-03-23 12:34:12.327] [driver] [info] [Task Finished] Transform to mem in 0.257823s (14232 normal rows, 0 exceptions)
[2021-03-23 12:34:12.584] [driver] [info] [Task Finished] Transform to mem in 0.257272s (14088 normal rows, 0 exceptions)
[2021-03-23 12:34:12.842] [driver] [info] [Task Finished] Transform to mem in 0.257663s (14106 normal rows, 0 exceptions)
[2021-03-23 12:34:13.100] [driver] [info] [Task Finished] Transform to mem in 0.257780s (14211 normal rows, 0 exceptions)
[2021-03-23 12:34:13.357] [driver] [info] [Task Finished] Transform to mem in 0.257258s (14113 normal rows, 0 exceptions)
[2021-03-23 12:34:13.614] [driver] [info] [Task Finished] Transform to mem in 0.257427s (14075 normal rows, 0 exceptions)
[2021-03-23 12:34:13.872] [driver] [info] [Task Finished] Transform to mem in 0.257793s (14223 normal rows, 0 exceptions)
[2021-03-23 12:34:14.129] [driver] [info] [Task Finished] Transform to mem in 0.257114s (14095 normal rows, 0 exceptions)
[2021-03-23 12:34:14.387] [driver] [info] [Task Finished] Transform to mem in 0.257683s (14086 normal rows, 0 exceptions)
[2021-03-23 12:34:14.645] [driver] [info] [Task Finished] Transform to mem in 0.257958s (14226 normal rows, 0 exceptions)
[2021-03-23 12:34:14.902] [driver] [info] [Task Finished] Transform to mem in 0.257496s (14170 normal rows, 0 exceptions)
[2021-03-23 12:34:15.160] [driver] [info] [Task Finished] Transform to mem in 0.257411s (14002 normal rows, 0 exceptions)
[2021-03-23 12:34:15.418] [driver] [info] [Task Finished] Transform to mem in 0.258092s (14226 normal rows, 0 exceptions)
[2021-03-23 12:34:15.676] [driver] [info] [Task Finished] Transform to mem in 0.257734s (14206 normal rows, 0 exceptions)
[2021-03-23 12:34:15.933] [driver] [info] [Task Finished] Transform to mem in 0.257053s (13970 normal rows, 0 exceptions)
[2021-03-23 12:34:16.192] [driver] [info] [Task Finished] Transform to mem in 0.258777s (14237 normal rows, 0 exceptions)
[2021-03-23 12:34:16.450] [driver] [info] [Task Finished] Transform to mem in 0.258630s (14233 normal rows, 0 exceptions)
[2021-03-23 12:34:16.707] [driver] [info] [Task Finished] Transform to mem in 0.257248s (13920 normal rows, 0 exceptions)
[2021-03-23 12:34:16.966] [driver] [info] [Task Finished] Transform to mem in 0.258094s (14267 normal rows, 0 exceptions)
[2021-03-23 12:34:17.223] [driver] [info] [Task Finished] Transform to mem in 0.257604s (14248 normal rows, 0 exceptions)
[2021-03-23 12:34:17.480] [driver] [info] [Task Finished] Transform to mem in 0.256629s (13912 normal rows, 0 exceptions)
[2021-03-23 12:34:17.738] [driver] [info] [Task Finished] Transform to mem in 0.258386s (14248 normal rows, 0 exceptions)
[2021-03-23 12:34:17.996] [driver] [info] [Task Finished] Transform to mem in 0.257634s (14237 normal rows, 0 exceptions)
[2021-03-23 12:34:18.253] [driver] [info] [Task Finished] Transform to mem in 0.257355s (13988 normal rows, 0 exceptions)
[2021-03-23 12:34:18.511] [driver] [info] [Task Finished] Transform to mem in 0.258154s (14171 normal rows, 0 exceptions)
[2021-03-23 12:34:18.770] [driver] [info] [Task Finished] Transform to mem in 0.258404s (14234 normal rows, 0 exceptions)
[2021-03-23 12:34:19.028] [driver] [info] [Task Finished] Transform to mem in 0.257871s (14015 normal rows, 0 exceptions)
[2021-03-23 12:34:19.286] [driver] [info] [Task Finished] Transform to mem in 0.258316s (14176 normal rows, 0 exceptions)
[2021-03-23 12:34:19.544] [driver] [info] [Task Finished] Transform to mem in 0.258405s (14246 normal rows, 0 exceptions)
[2021-03-23 12:34:19.802] [driver] [info] [Task Finished] Transform to mem in 0.257776s (14059 normal rows, 0 exceptions)
[2021-03-23 12:34:20.061] [driver] [info] [Task Finished] Transform to mem in 0.258507s (14114 normal rows, 0 exceptions)
[2021-03-23 12:34:20.318] [driver] [info] [Task Finished] Transform to mem in 0.257567s (14238 normal rows, 0 exceptions)
[2021-03-23 12:34:20.576] [driver] [info] [Task Finished] Transform to mem in 0.257777s (14081 normal rows, 0 exceptions)
[2021-03-23 12:34:20.835] [driver] [info] [Task Finished] Transform to mem in 0.258464s (14095 normal rows, 0 exceptions)
[2021-03-23 12:34:21.093] [driver] [info] [Task Finished] Transform to mem in 0.258073s (14221 normal rows, 0 exceptions)
[2021-03-23 12:34:21.351] [driver] [info] [Task Finished] Transform to mem in 0.257910s (14086 normal rows, 0 exceptions)
[2021-03-23 12:34:21.608] [driver] [info] [Task Finished] Transform to mem in 0.257659s (14091 normal rows, 0 exceptions)
[2021-03-23 12:34:21.866] [driver] [info] [Task Finished] Transform to mem in 0.257722s (14218 normal rows, 0 exceptions)
[2021-03-23 12:34:22.124] [driver] [info] [Task Finished] Transform to mem in 0.257611s (14095 normal rows, 0 exceptions)
[2021-03-23 12:34:22.381] [driver] [info] [Task Finished] Transform to mem in 0.257460s (14113 normal rows, 0 exceptions)
[2021-03-23 12:34:22.639] [driver] [info] [Task Finished] Transform to mem in 0.257630s (14197 normal rows, 0 exceptions)
[2021-03-23 12:34:22.896] [driver] [info] [Task Finished] Transform to mem in 0.257217s (14114 normal rows, 0 exceptions)
[2021-03-23 12:34:23.154] [driver] [info] [Task Finished] Transform to mem in 0.257980s (14094 normal rows, 0 exceptions)
[2021-03-23 12:34:23.413] [driver] [info] [Task Finished] Transform to mem in 0.258726s (14197 normal rows, 0 exceptions)
[2021-03-23 12:34:23.671] [driver] [info] [Task Finished] Transform to mem in 0.257963s (14115 normal rows, 0 exceptions)
[2021-03-23 12:34:23.929] [driver] [info] [Task Finished] Transform to mem in 0.258409s (14083 normal rows, 0 exceptions)
[2021-03-23 12:34:24.187] [driver] [info] [Task Finished] Transform to mem in 0.258255s (14235 normal rows, 0 exceptions)
[2021-03-23 12:34:24.446] [driver] [info] [Task Finished] Transform to mem in 0.258702s (14144 normal rows, 0 exceptions)
[2021-03-23 12:34:24.704] [driver] [info] [Task Finished] Transform to mem in 0.258002s (14015 normal rows, 0 exceptions)
[2021-03-23 12:34:24.964] [driver] [info] [Task Finished] Transform to mem in 0.259312s (14237 normal rows, 0 exceptions)
[2021-03-23 12:34:25.223] [driver] [info] [Task Finished] Transform to mem in 0.259145s (14159 normal rows, 0 exceptions)
[2021-03-23 12:34:25.482] [driver] [info] [Task Finished] Transform to mem in 0.258981s (14010 normal rows, 0 exceptions)
[2021-03-23 12:34:25.741] [driver] [info] [Task Finished] Transform to mem in 0.259178s (14246 normal rows, 0 exceptions)
[2021-03-23 12:34:25.999] [driver] [info] [Task Finished] Transform to mem in 0.258525s (14178 normal rows, 0 exceptions)
[2021-03-23 12:34:26.258] [driver] [info] [Task Finished] Transform to mem in 0.258553s (13970 normal rows, 0 exceptions)
[2021-03-23 12:34:26.517] [driver] [info] [Task Finished] Transform to mem in 0.259251s (14253 normal rows, 0 exceptions)
[2021-03-23 12:34:26.776] [driver] [info] [Task Finished] Transform to mem in 0.258754s (14236 normal rows, 0 exceptions)
[2021-03-23 12:34:27.035] [driver] [info] [Task Finished] Transform to mem in 0.258535s (13901 normal rows, 0 exceptions)
[2021-03-23 12:34:27.294] [driver] [info] [Task Finished] Transform to mem in 0.259049s (14267 normal rows, 0 exceptions)
[2021-03-23 12:34:27.552] [driver] [info] [Task Finished] Transform to mem in 0.258471s (14248 normal rows, 0 exceptions)
[2021-03-23 12:34:27.810] [driver] [info] [Task Finished] Transform to mem in 0.257647s (13953 normal rows, 0 exceptions)
[2021-03-23 12:34:28.069] [driver] [info] [Task Finished] Transform to mem in 0.258739s (14207 normal rows, 0 exceptions)
[2021-03-23 12:34:28.327] [driver] [info] [Task Finished] Transform to mem in 0.258495s (14245 normal rows, 0 exceptions)
[2021-03-23 12:34:28.585] [driver] [info] [Task Finished] Transform to mem in 0.257886s (13999 normal rows, 0 exceptions)
[2021-03-23 12:34:28.845] [driver] [info] [Task Finished] Transform to mem in 0.259886s (14158 normal rows, 0 exceptions)
[2021-03-23 12:34:29.103] [driver] [info] [Task Finished] Transform to mem in 0.257802s (14242 normal rows, 0 exceptions)
[2021-03-23 12:34:29.361] [driver] [info] [Task Finished] Transform to mem in 0.258100s (14020 normal rows, 0 exceptions)
[2021-03-23 12:34:29.620] [driver] [info] [Task Finished] Transform to mem in 0.258821s (14173 normal rows, 0 exceptions)
[2021-03-23 12:34:29.878] [driver] [info] [Task Finished] Transform to mem in 0.258053s (14235 normal rows, 0 exceptions)
[2021-03-23 12:34:30.136] [driver] [info] [Task Finished] Transform to mem in 0.258222s (14054 normal rows, 0 exceptions)
[2021-03-23 12:34:30.395] [driver] [info] [Task Finished] Transform to mem in 0.258975s (14114 normal rows, 0 exceptions)
[2021-03-23 12:34:30.653] [driver] [info] [Task Finished] Transform to mem in 0.258490s (14238 normal rows, 0 exceptions)
[2021-03-23 12:34:30.912] [driver] [info] [Task Finished] Transform to mem in 0.258046s (14081 normal rows, 0 exceptions)
[2021-03-23 12:34:31.169] [driver] [info] [Task Finished] Transform to mem in 0.257905s (14095 normal rows, 0 exceptions)
[2021-03-23 12:34:31.428] [driver] [info] [Task Finished] Transform to mem in 0.258663s (14221 normal rows, 0 exceptions)
[2021-03-23 12:34:31.686] [driver] [info] [Task Finished] Transform to mem in 0.258101s (14089 normal rows, 0 exceptions)
[2021-03-23 12:34:31.945] [driver] [info] [Task Finished] Transform to mem in 0.258283s (14094 normal rows, 0 exceptions)
[2021-03-23 12:34:32.203] [driver] [info] [Task Finished] Transform to mem in 0.258903s (14212 normal rows, 0 exceptions)
[2021-03-23 12:34:32.461] [driver] [info] [Task Finished] Transform to mem in 0.257609s (14108 normal rows, 0 exceptions)
[2021-03-23 12:34:32.719] [driver] [info] [Task Finished] Transform to mem in 0.258211s (14100 normal rows, 0 exceptions)
[2021-03-23 12:34:32.979] [driver] [info] [Task Finished] Transform to mem in 0.259180s (14217 normal rows, 0 exceptions)
[2021-03-23 12:34:33.237] [driver] [info] [Task Finished] Transform to mem in 0.258251s (14094 normal rows, 0 exceptions)
[2021-03-23 12:34:33.495] [driver] [info] [Task Finished] Transform to mem in 0.258371s (14094 normal rows, 0 exceptions)
[2021-03-23 12:34:33.754] [driver] [info] [Task Finished] Transform to mem in 0.258689s (14223 normal rows, 0 exceptions)
[2021-03-23 12:34:34.012] [driver] [info] [Task Finished] Transform to mem in 0.258463s (14089 normal rows, 0 exceptions)
[2021-03-23 12:34:34.270] [driver] [info] [Task Finished] Transform to mem in 0.258111s (14086 normal rows, 0 exceptions)
[2021-03-23 12:34:34.530] [driver] [info] [Task Finished] Transform to mem in 0.259310s (14232 normal rows, 0 exceptions)
[2021-03-23 12:34:34.790] [driver] [info] [Task Finished] Transform to mem in 0.259762s (14170 normal rows, 0 exceptions)
[2021-03-23 12:34:35.049] [driver] [info] [Task Finished] Transform to mem in 0.259236s (14002 normal rows, 0 exceptions)
[2021-03-23 12:34:35.309] [driver] [info] [Task Finished] Transform to mem in 0.259813s (14226 normal rows, 0 exceptions)
[2021-03-23 12:34:35.568] [driver] [info] [Task Finished] Transform to mem in 0.259557s (14206 normal rows, 0 exceptions)
[2021-03-23 12:34:35.827] [driver] [info] [Task Finished] Transform to mem in 0.259019s (13961 normal rows, 0 exceptions)
[2021-03-23 12:34:36.086] [driver] [info] [Task Finished] Transform to mem in 0.258801s (14246 normal rows, 0 exceptions)
[2021-03-23 12:34:36.346] [driver] [info] [Task Finished] Transform to mem in 0.259491s (14233 normal rows, 0 exceptions)
[2021-03-23 12:34:36.648] [driver] [info] [Task Finished] Transform to mem in 0.302041s (16320 normal rows, 0 exceptions)
[2021-03-23 12:34:36.648] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 41.3517s
[2021-03-23 12:34:36.648] [global] [info] [Transform Stage] Stage 0 total wall clock time: 41.3488s, 54108450 input rows, time to process 1 row via fast path: 0.000764184ms

[2021-03-23 12:34:36.689] [global] [info] Writing 2264570 rows as output to file (344.74 MB)
[2021-03-23 12:34:37.073] [driver] [info] [Task Finished] write to file in 0.384237s (2264570 rows, 344.74 MB)
[2021-03-23 12:34:37.073] [global] [info] writing output took 0.425314s
[2021-03-23 12:34:37.073] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.425401s
[2021-03-23 12:34:37.073] [global] [info] [Transform Stage] Stage 0 took 41.9844s
[2021-03-23 12:34:37.073] [global] [info] Query Execution took 42.0422s. (planning: 0.0577433s, execution: 41.9844s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 23 2021, 11:08:57) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z2_preprocessed/zillow_Z2_10G.csv']
Tuplex startup time: 0.004998207092285156
Tuplex job time: 42.30883026123047 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 0, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '32G', 'tuplex.env.hostname': '4e1bb76d17c5', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '4MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.0199255,"llvm_optimization_time_s":0.138056,"llvm_compilation_time_s":0.0617447,"total_compilation_time_s":0.207182,"sampling_time_s":0.110556,"stages":[{"stage_no":0,"fast_path_wall_time_s":41.3488,"fast_path_time_s":41.3518,"fast_path_per_row_time_ns":764.184,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.004998207092285156, "jobTime": 42.30883026123047}
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
