[2021-03-31 18:26:50.902] [global] [error] could not find backend, defaulting to local
[2021-03-31 18:26:50.903] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-31 18:26:50.903] [local ee] [info] initializing LLVM backend
[2021-03-31 18:26:50.903] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-31 18:26:50.903] [LLVM] [info] compiling code for skylake-avx512
[2021-03-31 18:26:50.904] [memory] [info] allocated bitmap managed memory region (32.00 GB, 32.00 MB block size)
[2021-03-31 18:26:50.904] [local execution engine] [info] started driver (32.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-31 18:26:50.904] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2021-03-31 18:26:50.904] [global] [info] sampled file:///data/zillow/Z2_preprocessed/zillow_Z2_10G.csv on 256.00 KB
[2021-03-31 18:26:51.036] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:51.051] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:51.059] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:51.085] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:51.105] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:51.118] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:51.149] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:51.165] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:26:51.167] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:26:51.169] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:26:51.172] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:26:51.187] [logical planner] [info] logical optimization took 0.019537ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-31 18:26:51.187] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (13 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:26:51.188] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-31 18:26:51.188] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-31 18:26:51.189] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-31 18:26:51.190] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-31 18:26:51.190] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-31 18:26:51.191] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-31 18:26:51.192] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> f64
[2021-03-31 18:26:51.193] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64)) -> i64
[2021-03-31 18:26:51.193] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64)) -> str
[2021-03-31 18:26:51.194] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64,str)) -> i64
[2021-03-31 18:26:51.195] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:26:51.196] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:26:51.196] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:26:51.222] [global] [info] Stage0 depends on: 
[2021-03-31 18:26:51.366] [global] [info] Optimization via LLVM passes took 0.136358 ms
[2021-03-31 18:26:51.427] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2021-03-31 18:26:51.427] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.205222s
[2021-03-31 18:26:51.427] [local ee] [info] split /data/zillow/Z2_preprocessed/zillow_Z2_10G.csv into 160 tasks
[2021-03-31 18:26:51.683] [driver] [info] [Task Finished] Transform to mem in 0.256051s (14226 normal rows, 0 exceptions)
[2021-03-31 18:26:51.937] [driver] [info] [Task Finished] Transform to mem in 0.253283s (14206 normal rows, 0 exceptions)
[2021-03-31 18:26:52.189] [driver] [info] [Task Finished] Transform to mem in 0.252652s (13970 normal rows, 0 exceptions)
[2021-03-31 18:26:52.443] [driver] [info] [Task Finished] Transform to mem in 0.253630s (14253 normal rows, 0 exceptions)
[2021-03-31 18:26:52.699] [driver] [info] [Task Finished] Transform to mem in 0.256156s (14217 normal rows, 0 exceptions)
[2021-03-31 18:26:52.955] [driver] [info] [Task Finished] Transform to mem in 0.255387s (13920 normal rows, 0 exceptions)
[2021-03-31 18:26:53.209] [driver] [info] [Task Finished] Transform to mem in 0.254091s (14267 normal rows, 0 exceptions)
[2021-03-31 18:26:53.462] [driver] [info] [Task Finished] Transform to mem in 0.253273s (14248 normal rows, 0 exceptions)
[2021-03-31 18:26:53.717] [driver] [info] [Task Finished] Transform to mem in 0.254899s (13953 normal rows, 0 exceptions)
[2021-03-31 18:26:53.972] [driver] [info] [Task Finished] Transform to mem in 0.255279s (14207 normal rows, 0 exceptions)
[2021-03-31 18:26:54.225] [driver] [info] [Task Finished] Transform to mem in 0.253165s (14237 normal rows, 0 exceptions)
[2021-03-31 18:26:54.478] [driver] [info] [Task Finished] Transform to mem in 0.252913s (13988 normal rows, 0 exceptions)
[2021-03-31 18:26:54.732] [driver] [info] [Task Finished] Transform to mem in 0.253606s (14177 normal rows, 0 exceptions)
[2021-03-31 18:26:54.985] [driver] [info] [Task Finished] Transform to mem in 0.253438s (14242 normal rows, 0 exceptions)
[2021-03-31 18:26:55.238] [driver] [info] [Task Finished] Transform to mem in 0.253002s (14020 normal rows, 0 exceptions)
[2021-03-31 18:26:55.492] [driver] [info] [Task Finished] Transform to mem in 0.253522s (14162 normal rows, 0 exceptions)
[2021-03-31 18:26:55.745] [driver] [info] [Task Finished] Transform to mem in 0.253237s (14241 normal rows, 0 exceptions)
[2021-03-31 18:26:55.998] [driver] [info] [Task Finished] Transform to mem in 0.252797s (14059 normal rows, 0 exceptions)
[2021-03-31 18:26:56.251] [driver] [info] [Task Finished] Transform to mem in 0.253208s (14114 normal rows, 0 exceptions)
[2021-03-31 18:26:56.508] [driver] [info] [Task Finished] Transform to mem in 0.256355s (14238 normal rows, 0 exceptions)
[2021-03-31 18:26:56.764] [driver] [info] [Task Finished] Transform to mem in 0.256233s (14081 normal rows, 0 exceptions)
[2021-03-31 18:26:57.017] [driver] [info] [Task Finished] Transform to mem in 0.253124s (14095 normal rows, 0 exceptions)
[2021-03-31 18:26:57.270] [driver] [info] [Task Finished] Transform to mem in 0.253218s (14221 normal rows, 0 exceptions)
[2021-03-31 18:26:57.523] [driver] [info] [Task Finished] Transform to mem in 0.253049s (14086 normal rows, 0 exceptions)
[2021-03-31 18:26:57.777] [driver] [info] [Task Finished] Transform to mem in 0.253243s (14097 normal rows, 0 exceptions)
[2021-03-31 18:26:58.030] [driver] [info] [Task Finished] Transform to mem in 0.253142s (14212 normal rows, 0 exceptions)
[2021-03-31 18:26:58.283] [driver] [info] [Task Finished] Transform to mem in 0.253235s (14108 normal rows, 0 exceptions)
[2021-03-31 18:26:58.536] [driver] [info] [Task Finished] Transform to mem in 0.253027s (14100 normal rows, 0 exceptions)
[2021-03-31 18:26:58.789] [driver] [info] [Task Finished] Transform to mem in 0.253079s (14212 normal rows, 0 exceptions)
[2021-03-31 18:26:59.043] [driver] [info] [Task Finished] Transform to mem in 0.253812s (14099 normal rows, 0 exceptions)
[2021-03-31 18:26:59.296] [driver] [info] [Task Finished] Transform to mem in 0.253461s (14094 normal rows, 0 exceptions)
[2021-03-31 18:26:59.550] [driver] [info] [Task Finished] Transform to mem in 0.253056s (14223 normal rows, 0 exceptions)
[2021-03-31 18:26:59.803] [driver] [info] [Task Finished] Transform to mem in 0.253173s (14089 normal rows, 0 exceptions)
[2021-03-31 18:27:00.059] [driver] [info] [Task Finished] Transform to mem in 0.256151s (14083 normal rows, 0 exceptions)
[2021-03-31 18:27:00.315] [driver] [info] [Task Finished] Transform to mem in 0.256288s (14235 normal rows, 0 exceptions)
[2021-03-31 18:27:00.568] [driver] [info] [Task Finished] Transform to mem in 0.253248s (14170 normal rows, 0 exceptions)
[2021-03-31 18:27:00.821] [driver] [info] [Task Finished] Transform to mem in 0.252897s (14002 normal rows, 0 exceptions)
[2021-03-31 18:27:01.075] [driver] [info] [Task Finished] Transform to mem in 0.253534s (14226 normal rows, 0 exceptions)
[2021-03-31 18:27:01.328] [driver] [info] [Task Finished] Transform to mem in 0.253142s (14206 normal rows, 0 exceptions)
[2021-03-31 18:27:01.581] [driver] [info] [Task Finished] Transform to mem in 0.253040s (13961 normal rows, 0 exceptions)
[2021-03-31 18:27:01.835] [driver] [info] [Task Finished] Transform to mem in 0.253561s (14246 normal rows, 0 exceptions)
[2021-03-31 18:27:02.088] [driver] [info] [Task Finished] Transform to mem in 0.253379s (14233 normal rows, 0 exceptions)
[2021-03-31 18:27:02.341] [driver] [info] [Task Finished] Transform to mem in 0.253031s (13915 normal rows, 0 exceptions)
[2021-03-31 18:27:02.595] [driver] [info] [Task Finished] Transform to mem in 0.253551s (14253 normal rows, 0 exceptions)
[2021-03-31 18:27:02.848] [driver] [info] [Task Finished] Transform to mem in 0.253273s (14267 normal rows, 0 exceptions)
[2021-03-31 18:27:03.101] [driver] [info] [Task Finished] Transform to mem in 0.252723s (13892 normal rows, 0 exceptions)
[2021-03-31 18:27:03.354] [driver] [info] [Task Finished] Transform to mem in 0.253432s (14245 normal rows, 0 exceptions)
[2021-03-31 18:27:03.607] [driver] [info] [Task Finished] Transform to mem in 0.253155s (14260 normal rows, 0 exceptions)
[2021-03-31 18:27:03.863] [driver] [info] [Task Finished] Transform to mem in 0.255743s (13978 normal rows, 0 exceptions)
[2021-03-31 18:27:04.120] [driver] [info] [Task Finished] Transform to mem in 0.256550s (14181 normal rows, 0 exceptions)
[2021-03-31 18:27:04.373] [driver] [info] [Task Finished] Transform to mem in 0.253371s (14234 normal rows, 0 exceptions)
[2021-03-31 18:27:04.626] [driver] [info] [Task Finished] Transform to mem in 0.252719s (14015 normal rows, 0 exceptions)
[2021-03-31 18:27:04.879] [driver] [info] [Task Finished] Transform to mem in 0.253094s (14142 normal rows, 0 exceptions)
[2021-03-31 18:27:05.132] [driver] [info] [Task Finished] Transform to mem in 0.253503s (14280 normal rows, 0 exceptions)
[2021-03-31 18:27:05.385] [driver] [info] [Task Finished] Transform to mem in 0.252818s (13982 normal rows, 0 exceptions)
[2021-03-31 18:27:05.639] [driver] [info] [Task Finished] Transform to mem in 0.253366s (14173 normal rows, 0 exceptions)
[2021-03-31 18:27:05.892] [driver] [info] [Task Finished] Transform to mem in 0.253383s (14235 normal rows, 0 exceptions)
[2021-03-31 18:27:06.145] [driver] [info] [Task Finished] Transform to mem in 0.252741s (14054 normal rows, 0 exceptions)
[2021-03-31 18:27:06.398] [driver] [info] [Task Finished] Transform to mem in 0.253350s (14129 normal rows, 0 exceptions)
[2021-03-31 18:27:06.652] [driver] [info] [Task Finished] Transform to mem in 0.253405s (14223 normal rows, 0 exceptions)
[2021-03-31 18:27:06.904] [driver] [info] [Task Finished] Transform to mem in 0.252635s (14081 normal rows, 0 exceptions)
[2021-03-31 18:27:07.157] [driver] [info] [Task Finished] Transform to mem in 0.253093s (14108 normal rows, 0 exceptions)
[2021-03-31 18:27:07.414] [driver] [info] [Task Finished] Transform to mem in 0.256365s (14208 normal rows, 0 exceptions)
[2021-03-31 18:27:07.666] [driver] [info] [Task Finished] Transform to mem in 0.252658s (14089 normal rows, 0 exceptions)
[2021-03-31 18:27:07.924] [driver] [info] [Task Finished] Transform to mem in 0.257265s (14094 normal rows, 0 exceptions)
[2021-03-31 18:27:08.176] [driver] [info] [Task Finished] Transform to mem in 0.252831s (14232 normal rows, 0 exceptions)
[2021-03-31 18:27:08.429] [driver] [info] [Task Finished] Transform to mem in 0.253007s (14088 normal rows, 0 exceptions)
[2021-03-31 18:27:08.683] [driver] [info] [Task Finished] Transform to mem in 0.253173s (14106 normal rows, 0 exceptions)
[2021-03-31 18:27:08.936] [driver] [info] [Task Finished] Transform to mem in 0.253339s (14211 normal rows, 0 exceptions)
[2021-03-31 18:27:09.189] [driver] [info] [Task Finished] Transform to mem in 0.252853s (14113 normal rows, 0 exceptions)
[2021-03-31 18:27:09.442] [driver] [info] [Task Finished] Transform to mem in 0.253062s (14075 normal rows, 0 exceptions)
[2021-03-31 18:27:09.695] [driver] [info] [Task Finished] Transform to mem in 0.253303s (14223 normal rows, 0 exceptions)
[2021-03-31 18:27:09.948] [driver] [info] [Task Finished] Transform to mem in 0.253068s (14095 normal rows, 0 exceptions)
[2021-03-31 18:27:10.201] [driver] [info] [Task Finished] Transform to mem in 0.253053s (14086 normal rows, 0 exceptions)
[2021-03-31 18:27:10.455] [driver] [info] [Task Finished] Transform to mem in 0.253456s (14226 normal rows, 0 exceptions)
[2021-03-31 18:27:10.708] [driver] [info] [Task Finished] Transform to mem in 0.252637s (14170 normal rows, 0 exceptions)
[2021-03-31 18:27:10.960] [driver] [info] [Task Finished] Transform to mem in 0.252612s (14002 normal rows, 0 exceptions)
[2021-03-31 18:27:11.217] [driver] [info] [Task Finished] Transform to mem in 0.256657s (14226 normal rows, 0 exceptions)
[2021-03-31 18:27:11.473] [driver] [info] [Task Finished] Transform to mem in 0.256390s (14206 normal rows, 0 exceptions)
[2021-03-31 18:27:11.726] [driver] [info] [Task Finished] Transform to mem in 0.252559s (13970 normal rows, 0 exceptions)
[2021-03-31 18:27:11.979] [driver] [info] [Task Finished] Transform to mem in 0.253251s (14237 normal rows, 0 exceptions)
[2021-03-31 18:27:12.233] [driver] [info] [Task Finished] Transform to mem in 0.253417s (14233 normal rows, 0 exceptions)
[2021-03-31 18:27:12.485] [driver] [info] [Task Finished] Transform to mem in 0.252631s (13920 normal rows, 0 exceptions)
[2021-03-31 18:27:12.739] [driver] [info] [Task Finished] Transform to mem in 0.253316s (14267 normal rows, 0 exceptions)
[2021-03-31 18:27:12.992] [driver] [info] [Task Finished] Transform to mem in 0.253476s (14248 normal rows, 0 exceptions)
[2021-03-31 18:27:13.245] [driver] [info] [Task Finished] Transform to mem in 0.252515s (13912 normal rows, 0 exceptions)
[2021-03-31 18:27:13.498] [driver] [info] [Task Finished] Transform to mem in 0.253560s (14248 normal rows, 0 exceptions)
[2021-03-31 18:27:13.752] [driver] [info] [Task Finished] Transform to mem in 0.253505s (14237 normal rows, 0 exceptions)
[2021-03-31 18:27:14.004] [driver] [info] [Task Finished] Transform to mem in 0.252488s (13988 normal rows, 0 exceptions)
[2021-03-31 18:27:14.258] [driver] [info] [Task Finished] Transform to mem in 0.253397s (14171 normal rows, 0 exceptions)
[2021-03-31 18:27:14.511] [driver] [info] [Task Finished] Transform to mem in 0.253182s (14234 normal rows, 0 exceptions)
[2021-03-31 18:27:14.767] [driver] [info] [Task Finished] Transform to mem in 0.255864s (14015 normal rows, 0 exceptions)
[2021-03-31 18:27:15.020] [driver] [info] [Task Finished] Transform to mem in 0.253062s (14176 normal rows, 0 exceptions)
[2021-03-31 18:27:15.276] [driver] [info] [Task Finished] Transform to mem in 0.256531s (14246 normal rows, 0 exceptions)
[2021-03-31 18:27:15.529] [driver] [info] [Task Finished] Transform to mem in 0.252781s (14059 normal rows, 0 exceptions)
[2021-03-31 18:27:15.782] [driver] [info] [Task Finished] Transform to mem in 0.253391s (14114 normal rows, 0 exceptions)
[2021-03-31 18:27:16.035] [driver] [info] [Task Finished] Transform to mem in 0.252982s (14238 normal rows, 0 exceptions)
[2021-03-31 18:27:16.289] [driver] [info] [Task Finished] Transform to mem in 0.253074s (14081 normal rows, 0 exceptions)
[2021-03-31 18:27:16.542] [driver] [info] [Task Finished] Transform to mem in 0.253410s (14095 normal rows, 0 exceptions)
[2021-03-31 18:27:16.795] [driver] [info] [Task Finished] Transform to mem in 0.253050s (14221 normal rows, 0 exceptions)
[2021-03-31 18:27:17.048] [driver] [info] [Task Finished] Transform to mem in 0.253055s (14086 normal rows, 0 exceptions)
[2021-03-31 18:27:17.301] [driver] [info] [Task Finished] Transform to mem in 0.253285s (14091 normal rows, 0 exceptions)
[2021-03-31 18:27:17.555] [driver] [info] [Task Finished] Transform to mem in 0.253217s (14218 normal rows, 0 exceptions)
[2021-03-31 18:27:17.808] [driver] [info] [Task Finished] Transform to mem in 0.253110s (14095 normal rows, 0 exceptions)
[2021-03-31 18:27:18.061] [driver] [info] [Task Finished] Transform to mem in 0.253215s (14113 normal rows, 0 exceptions)
[2021-03-31 18:27:18.314] [driver] [info] [Task Finished] Transform to mem in 0.253214s (14197 normal rows, 0 exceptions)
[2021-03-31 18:27:18.570] [driver] [info] [Task Finished] Transform to mem in 0.256045s (14114 normal rows, 0 exceptions)
[2021-03-31 18:27:18.827] [driver] [info] [Task Finished] Transform to mem in 0.256853s (14094 normal rows, 0 exceptions)
[2021-03-31 18:27:19.081] [driver] [info] [Task Finished] Transform to mem in 0.253795s (14197 normal rows, 0 exceptions)
[2021-03-31 18:27:19.334] [driver] [info] [Task Finished] Transform to mem in 0.253060s (14115 normal rows, 0 exceptions)
[2021-03-31 18:27:19.588] [driver] [info] [Task Finished] Transform to mem in 0.253863s (14083 normal rows, 0 exceptions)
[2021-03-31 18:27:19.841] [driver] [info] [Task Finished] Transform to mem in 0.253572s (14235 normal rows, 0 exceptions)
[2021-03-31 18:27:20.094] [driver] [info] [Task Finished] Transform to mem in 0.252680s (14144 normal rows, 0 exceptions)
[2021-03-31 18:27:20.347] [driver] [info] [Task Finished] Transform to mem in 0.252929s (14015 normal rows, 0 exceptions)
[2021-03-31 18:27:20.601] [driver] [info] [Task Finished] Transform to mem in 0.253578s (14237 normal rows, 0 exceptions)
[2021-03-31 18:27:20.854] [driver] [info] [Task Finished] Transform to mem in 0.253291s (14159 normal rows, 0 exceptions)
[2021-03-31 18:27:21.107] [driver] [info] [Task Finished] Transform to mem in 0.252734s (14010 normal rows, 0 exceptions)
[2021-03-31 18:27:21.360] [driver] [info] [Task Finished] Transform to mem in 0.253064s (14246 normal rows, 0 exceptions)
[2021-03-31 18:27:21.613] [driver] [info] [Task Finished] Transform to mem in 0.252981s (14178 normal rows, 0 exceptions)
[2021-03-31 18:27:21.865] [driver] [info] [Task Finished] Transform to mem in 0.252310s (13970 normal rows, 0 exceptions)
[2021-03-31 18:27:22.122] [driver] [info] [Task Finished] Transform to mem in 0.256747s (14253 normal rows, 0 exceptions)
[2021-03-31 18:27:22.375] [driver] [info] [Task Finished] Transform to mem in 0.252728s (14236 normal rows, 0 exceptions)
[2021-03-31 18:27:22.630] [driver] [info] [Task Finished] Transform to mem in 0.255479s (13901 normal rows, 0 exceptions)
[2021-03-31 18:27:22.884] [driver] [info] [Task Finished] Transform to mem in 0.253514s (14267 normal rows, 0 exceptions)
[2021-03-31 18:27:23.137] [driver] [info] [Task Finished] Transform to mem in 0.253046s (14248 normal rows, 0 exceptions)
[2021-03-31 18:27:23.389] [driver] [info] [Task Finished] Transform to mem in 0.252171s (13953 normal rows, 0 exceptions)
[2021-03-31 18:27:23.645] [driver] [info] [Task Finished] Transform to mem in 0.256551s (14207 normal rows, 0 exceptions)
[2021-03-31 18:27:23.899] [driver] [info] [Task Finished] Transform to mem in 0.253012s (14245 normal rows, 0 exceptions)
[2021-03-31 18:27:24.151] [driver] [info] [Task Finished] Transform to mem in 0.252686s (13999 normal rows, 0 exceptions)
[2021-03-31 18:27:24.404] [driver] [info] [Task Finished] Transform to mem in 0.253241s (14158 normal rows, 0 exceptions)
[2021-03-31 18:27:24.657] [driver] [info] [Task Finished] Transform to mem in 0.252864s (14242 normal rows, 0 exceptions)
[2021-03-31 18:27:24.910] [driver] [info] [Task Finished] Transform to mem in 0.252528s (14020 normal rows, 0 exceptions)
[2021-03-31 18:27:25.163] [driver] [info] [Task Finished] Transform to mem in 0.253329s (14173 normal rows, 0 exceptions)
[2021-03-31 18:27:25.416] [driver] [info] [Task Finished] Transform to mem in 0.252779s (14235 normal rows, 0 exceptions)
[2021-03-31 18:27:25.669] [driver] [info] [Task Finished] Transform to mem in 0.252630s (14054 normal rows, 0 exceptions)
[2021-03-31 18:27:25.925] [driver] [info] [Task Finished] Transform to mem in 0.256392s (14114 normal rows, 0 exceptions)
[2021-03-31 18:27:26.181] [driver] [info] [Task Finished] Transform to mem in 0.255688s (14238 normal rows, 0 exceptions)
[2021-03-31 18:27:26.434] [driver] [info] [Task Finished] Transform to mem in 0.253636s (14081 normal rows, 0 exceptions)
[2021-03-31 18:27:26.687] [driver] [info] [Task Finished] Transform to mem in 0.252697s (14095 normal rows, 0 exceptions)
[2021-03-31 18:27:26.940] [driver] [info] [Task Finished] Transform to mem in 0.253207s (14221 normal rows, 0 exceptions)
[2021-03-31 18:27:27.193] [driver] [info] [Task Finished] Transform to mem in 0.252816s (14089 normal rows, 0 exceptions)
[2021-03-31 18:27:27.446] [driver] [info] [Task Finished] Transform to mem in 0.253075s (14094 normal rows, 0 exceptions)
[2021-03-31 18:27:27.699] [driver] [info] [Task Finished] Transform to mem in 0.253005s (14212 normal rows, 0 exceptions)
[2021-03-31 18:27:27.952] [driver] [info] [Task Finished] Transform to mem in 0.252999s (14108 normal rows, 0 exceptions)
[2021-03-31 18:27:28.205] [driver] [info] [Task Finished] Transform to mem in 0.253067s (14100 normal rows, 0 exceptions)
[2021-03-31 18:27:28.459] [driver] [info] [Task Finished] Transform to mem in 0.253276s (14217 normal rows, 0 exceptions)
[2021-03-31 18:27:28.712] [driver] [info] [Task Finished] Transform to mem in 0.253029s (14094 normal rows, 0 exceptions)
[2021-03-31 18:27:28.965] [driver] [info] [Task Finished] Transform to mem in 0.252843s (14094 normal rows, 0 exceptions)
[2021-03-31 18:27:29.218] [driver] [info] [Task Finished] Transform to mem in 0.253100s (14223 normal rows, 0 exceptions)
[2021-03-31 18:27:29.474] [driver] [info] [Task Finished] Transform to mem in 0.256130s (14089 normal rows, 0 exceptions)
[2021-03-31 18:27:29.726] [driver] [info] [Task Finished] Transform to mem in 0.252444s (14086 normal rows, 0 exceptions)
[2021-03-31 18:27:29.983] [driver] [info] [Task Finished] Transform to mem in 0.256704s (14232 normal rows, 0 exceptions)
[2021-03-31 18:27:30.236] [driver] [info] [Task Finished] Transform to mem in 0.252856s (14170 normal rows, 0 exceptions)
[2021-03-31 18:27:30.489] [driver] [info] [Task Finished] Transform to mem in 0.252601s (14002 normal rows, 0 exceptions)
[2021-03-31 18:27:30.742] [driver] [info] [Task Finished] Transform to mem in 0.253439s (14226 normal rows, 0 exceptions)
[2021-03-31 18:27:30.995] [driver] [info] [Task Finished] Transform to mem in 0.253026s (14206 normal rows, 0 exceptions)
[2021-03-31 18:27:31.248] [driver] [info] [Task Finished] Transform to mem in 0.252589s (13961 normal rows, 0 exceptions)
[2021-03-31 18:27:31.501] [driver] [info] [Task Finished] Transform to mem in 0.253492s (14246 normal rows, 0 exceptions)
[2021-03-31 18:27:31.754] [driver] [info] [Task Finished] Transform to mem in 0.253095s (14233 normal rows, 0 exceptions)
[2021-03-31 18:27:32.049] [driver] [info] [Task Finished] Transform to mem in 0.294574s (16320 normal rows, 0 exceptions)
[2021-03-31 18:27:32.049] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 40.6218s
[2021-03-31 18:27:32.049] [global] [info] [Transform Stage] Stage 0 total wall clock time: 40.6191s, 54108450 input rows, time to process 1 row via fast path: 0.000750697ms

[2021-03-31 18:27:32.086] [global] [info] Writing 2264570 rows as output to file (344.74 MB)
[2021-03-31 18:27:32.504] [driver] [info] [Task Finished] write to file in 0.417299s (2264570 rows, 344.74 MB)
[2021-03-31 18:27:32.504] [global] [info] writing output took 0.454735s
[2021-03-31 18:27:32.504] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.454833s
[2021-03-31 18:27:32.504] [global] [info] [Transform Stage] Stage 0 took 41.2819s
[2021-03-31 18:27:32.504] [global] [info] Query Execution took 41.3387s. (planning: 0.0568194s, execution: 41.2819s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 31 2021, 05:47:10) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z2_preprocessed/zillow_Z2_10G.csv']
Tuplex startup time: 0.0050640106201171875
Tuplex job time: 41.60131359100342 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 0, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '32G', 'tuplex.env.hostname': '6bcf8f6a24af', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '8MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.019537,"llvm_optimization_time_s":0.136358,"llvm_compilation_time_s":0.061463,"total_compilation_time_s":0.20522,"sampling_time_s":0.108658,"stages":[{"stage_no":0,"fast_path_wall_time_s":40.6191,"fast_path_time_s":40.6218,"fast_path_per_row_time_ns":750.697,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.0050640106201171875, "jobTime": 41.60131359100342}
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
