[2021-03-31 18:26:08.992] [global] [error] could not find backend, defaulting to local
[2021-03-31 18:26:08.993] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-31 18:26:08.993] [local ee] [info] initializing LLVM backend
[2021-03-31 18:26:08.993] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-31 18:26:08.993] [LLVM] [info] compiling code for skylake-avx512
[2021-03-31 18:26:08.993] [memory] [info] allocated bitmap managed memory region (32.00 GB, 32.00 MB block size)
[2021-03-31 18:26:08.993] [local execution engine] [info] started driver (32.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-31 18:26:08.994] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2021-03-31 18:26:08.994] [global] [info] sampled file:///data/zillow/Z2_preprocessed/zillow_Z2_10G.csv on 256.00 KB
[2021-03-31 18:26:09.132] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:09.147] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:09.155] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:09.181] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:09.201] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:09.214] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:09.245] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:26:09.261] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:26:09.263] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:26:09.265] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:26:09.268] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:26:09.283] [logical planner] [info] logical optimization took 0.019586ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-31 18:26:09.283] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (13 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:26:09.284] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-31 18:26:09.284] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-31 18:26:09.285] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-31 18:26:09.286] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-31 18:26:09.286] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-31 18:26:09.287] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-31 18:26:09.288] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> f64
[2021-03-31 18:26:09.288] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64)) -> i64
[2021-03-31 18:26:09.289] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64)) -> str
[2021-03-31 18:26:09.290] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64,str)) -> i64
[2021-03-31 18:26:09.291] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:26:09.292] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:26:09.292] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:26:09.317] [global] [info] Stage0 depends on: 
[2021-03-31 18:26:09.460] [global] [info] Optimization via LLVM passes took 0.135517 ms
[2021-03-31 18:26:09.521] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2021-03-31 18:26:09.521] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.203952s
[2021-03-31 18:26:09.522] [local ee] [info] split /data/zillow/Z2_preprocessed/zillow_Z2_10G.csv into 160 tasks
[2021-03-31 18:26:09.777] [driver] [info] [Task Finished] Transform to mem in 0.255453s (14226 normal rows, 0 exceptions)
[2021-03-31 18:26:10.030] [driver] [info] [Task Finished] Transform to mem in 0.252922s (14206 normal rows, 0 exceptions)
[2021-03-31 18:26:10.282] [driver] [info] [Task Finished] Transform to mem in 0.252285s (13970 normal rows, 0 exceptions)
[2021-03-31 18:26:10.536] [driver] [info] [Task Finished] Transform to mem in 0.253315s (14253 normal rows, 0 exceptions)
[2021-03-31 18:26:10.788] [driver] [info] [Task Finished] Transform to mem in 0.252613s (14217 normal rows, 0 exceptions)
[2021-03-31 18:26:11.041] [driver] [info] [Task Finished] Transform to mem in 0.252605s (13920 normal rows, 0 exceptions)
[2021-03-31 18:26:11.294] [driver] [info] [Task Finished] Transform to mem in 0.253416s (14267 normal rows, 0 exceptions)
[2021-03-31 18:26:11.547] [driver] [info] [Task Finished] Transform to mem in 0.252689s (14248 normal rows, 0 exceptions)
[2021-03-31 18:26:11.799] [driver] [info] [Task Finished] Transform to mem in 0.252172s (13953 normal rows, 0 exceptions)
[2021-03-31 18:26:12.052] [driver] [info] [Task Finished] Transform to mem in 0.253112s (14207 normal rows, 0 exceptions)
[2021-03-31 18:26:12.305] [driver] [info] [Task Finished] Transform to mem in 0.252679s (14237 normal rows, 0 exceptions)
[2021-03-31 18:26:12.561] [driver] [info] [Task Finished] Transform to mem in 0.255758s (13988 normal rows, 0 exceptions)
[2021-03-31 18:26:12.814] [driver] [info] [Task Finished] Transform to mem in 0.252896s (14177 normal rows, 0 exceptions)
[2021-03-31 18:26:13.067] [driver] [info] [Task Finished] Transform to mem in 0.253117s (14242 normal rows, 0 exceptions)
[2021-03-31 18:26:13.320] [driver] [info] [Task Finished] Transform to mem in 0.252655s (14020 normal rows, 0 exceptions)
[2021-03-31 18:26:13.573] [driver] [info] [Task Finished] Transform to mem in 0.253026s (14162 normal rows, 0 exceptions)
[2021-03-31 18:26:13.825] [driver] [info] [Task Finished] Transform to mem in 0.252897s (14241 normal rows, 0 exceptions)
[2021-03-31 18:26:14.078] [driver] [info] [Task Finished] Transform to mem in 0.252762s (14059 normal rows, 0 exceptions)
[2021-03-31 18:26:14.331] [driver] [info] [Task Finished] Transform to mem in 0.253044s (14114 normal rows, 0 exceptions)
[2021-03-31 18:26:14.584] [driver] [info] [Task Finished] Transform to mem in 0.252757s (14238 normal rows, 0 exceptions)
[2021-03-31 18:26:14.836] [driver] [info] [Task Finished] Transform to mem in 0.252401s (14081 normal rows, 0 exceptions)
[2021-03-31 18:26:15.089] [driver] [info] [Task Finished] Transform to mem in 0.252682s (14095 normal rows, 0 exceptions)
[2021-03-31 18:26:15.342] [driver] [info] [Task Finished] Transform to mem in 0.253022s (14221 normal rows, 0 exceptions)
[2021-03-31 18:26:15.595] [driver] [info] [Task Finished] Transform to mem in 0.252485s (14086 normal rows, 0 exceptions)
[2021-03-31 18:26:15.847] [driver] [info] [Task Finished] Transform to mem in 0.252684s (14097 normal rows, 0 exceptions)
[2021-03-31 18:26:16.100] [driver] [info] [Task Finished] Transform to mem in 0.252889s (14212 normal rows, 0 exceptions)
[2021-03-31 18:26:16.356] [driver] [info] [Task Finished] Transform to mem in 0.255948s (14108 normal rows, 0 exceptions)
[2021-03-31 18:26:16.609] [driver] [info] [Task Finished] Transform to mem in 0.252545s (14100 normal rows, 0 exceptions)
[2021-03-31 18:26:16.862] [driver] [info] [Task Finished] Transform to mem in 0.253063s (14212 normal rows, 0 exceptions)
[2021-03-31 18:26:17.114] [driver] [info] [Task Finished] Transform to mem in 0.252442s (14099 normal rows, 0 exceptions)
[2021-03-31 18:26:17.367] [driver] [info] [Task Finished] Transform to mem in 0.252463s (14094 normal rows, 0 exceptions)
[2021-03-31 18:26:17.620] [driver] [info] [Task Finished] Transform to mem in 0.252820s (14223 normal rows, 0 exceptions)
[2021-03-31 18:26:17.872] [driver] [info] [Task Finished] Transform to mem in 0.252513s (14089 normal rows, 0 exceptions)
[2021-03-31 18:26:18.125] [driver] [info] [Task Finished] Transform to mem in 0.252675s (14083 normal rows, 0 exceptions)
[2021-03-31 18:26:18.378] [driver] [info] [Task Finished] Transform to mem in 0.253179s (14235 normal rows, 0 exceptions)
[2021-03-31 18:26:18.631] [driver] [info] [Task Finished] Transform to mem in 0.252479s (14170 normal rows, 0 exceptions)
[2021-03-31 18:26:18.884] [driver] [info] [Task Finished] Transform to mem in 0.253204s (14002 normal rows, 0 exceptions)
[2021-03-31 18:26:19.137] [driver] [info] [Task Finished] Transform to mem in 0.252979s (14226 normal rows, 0 exceptions)
[2021-03-31 18:26:19.389] [driver] [info] [Task Finished] Transform to mem in 0.252409s (14206 normal rows, 0 exceptions)
[2021-03-31 18:26:19.641] [driver] [info] [Task Finished] Transform to mem in 0.252277s (13961 normal rows, 0 exceptions)
[2021-03-31 18:26:19.898] [driver] [info] [Task Finished] Transform to mem in 0.256178s (14246 normal rows, 0 exceptions)
[2021-03-31 18:26:20.150] [driver] [info] [Task Finished] Transform to mem in 0.252713s (14233 normal rows, 0 exceptions)
[2021-03-31 18:26:20.403] [driver] [info] [Task Finished] Transform to mem in 0.252148s (13915 normal rows, 0 exceptions)
[2021-03-31 18:26:20.656] [driver] [info] [Task Finished] Transform to mem in 0.253229s (14253 normal rows, 0 exceptions)
[2021-03-31 18:26:20.909] [driver] [info] [Task Finished] Transform to mem in 0.252845s (14267 normal rows, 0 exceptions)
[2021-03-31 18:26:21.161] [driver] [info] [Task Finished] Transform to mem in 0.252116s (13892 normal rows, 0 exceptions)
[2021-03-31 18:26:21.414] [driver] [info] [Task Finished] Transform to mem in 0.253262s (14245 normal rows, 0 exceptions)
[2021-03-31 18:26:21.667] [driver] [info] [Task Finished] Transform to mem in 0.252684s (14260 normal rows, 0 exceptions)
[2021-03-31 18:26:21.919] [driver] [info] [Task Finished] Transform to mem in 0.252120s (13978 normal rows, 0 exceptions)
[2021-03-31 18:26:22.173] [driver] [info] [Task Finished] Transform to mem in 0.253625s (14181 normal rows, 0 exceptions)
[2021-03-31 18:26:22.425] [driver] [info] [Task Finished] Transform to mem in 0.252729s (14234 normal rows, 0 exceptions)
[2021-03-31 18:26:22.678] [driver] [info] [Task Finished] Transform to mem in 0.252360s (14015 normal rows, 0 exceptions)
[2021-03-31 18:26:22.931] [driver] [info] [Task Finished] Transform to mem in 0.252993s (14142 normal rows, 0 exceptions)
[2021-03-31 18:26:23.187] [driver] [info] [Task Finished] Transform to mem in 0.256188s (14280 normal rows, 0 exceptions)
[2021-03-31 18:26:23.439] [driver] [info] [Task Finished] Transform to mem in 0.252466s (13982 normal rows, 0 exceptions)
[2021-03-31 18:26:23.696] [driver] [info] [Task Finished] Transform to mem in 0.256343s (14173 normal rows, 0 exceptions)
[2021-03-31 18:26:23.949] [driver] [info] [Task Finished] Transform to mem in 0.252876s (14235 normal rows, 0 exceptions)
[2021-03-31 18:26:24.201] [driver] [info] [Task Finished] Transform to mem in 0.252715s (14054 normal rows, 0 exceptions)
[2021-03-31 18:26:24.454] [driver] [info] [Task Finished] Transform to mem in 0.252728s (14129 normal rows, 0 exceptions)
[2021-03-31 18:26:24.707] [driver] [info] [Task Finished] Transform to mem in 0.252883s (14223 normal rows, 0 exceptions)
[2021-03-31 18:26:24.959] [driver] [info] [Task Finished] Transform to mem in 0.252465s (14081 normal rows, 0 exceptions)
[2021-03-31 18:26:25.212] [driver] [info] [Task Finished] Transform to mem in 0.252958s (14108 normal rows, 0 exceptions)
[2021-03-31 18:26:25.465] [driver] [info] [Task Finished] Transform to mem in 0.252892s (14208 normal rows, 0 exceptions)
[2021-03-31 18:26:25.718] [driver] [info] [Task Finished] Transform to mem in 0.252417s (14089 normal rows, 0 exceptions)
[2021-03-31 18:26:25.970] [driver] [info] [Task Finished] Transform to mem in 0.252592s (14094 normal rows, 0 exceptions)
[2021-03-31 18:26:26.223] [driver] [info] [Task Finished] Transform to mem in 0.252926s (14232 normal rows, 0 exceptions)
[2021-03-31 18:26:26.476] [driver] [info] [Task Finished] Transform to mem in 0.252333s (14088 normal rows, 0 exceptions)
[2021-03-31 18:26:26.728] [driver] [info] [Task Finished] Transform to mem in 0.252385s (14106 normal rows, 0 exceptions)
[2021-03-31 18:26:26.984] [driver] [info] [Task Finished] Transform to mem in 0.256041s (14211 normal rows, 0 exceptions)
[2021-03-31 18:26:27.240] [driver] [info] [Task Finished] Transform to mem in 0.255769s (14113 normal rows, 0 exceptions)
[2021-03-31 18:26:27.493] [driver] [info] [Task Finished] Transform to mem in 0.253597s (14075 normal rows, 0 exceptions)
[2021-03-31 18:26:27.747] [driver] [info] [Task Finished] Transform to mem in 0.253202s (14223 normal rows, 0 exceptions)
[2021-03-31 18:26:27.999] [driver] [info] [Task Finished] Transform to mem in 0.252770s (14095 normal rows, 0 exceptions)
[2021-03-31 18:26:28.252] [driver] [info] [Task Finished] Transform to mem in 0.252996s (14086 normal rows, 0 exceptions)
[2021-03-31 18:26:28.506] [driver] [info] [Task Finished] Transform to mem in 0.253660s (14226 normal rows, 0 exceptions)
[2021-03-31 18:26:28.760] [driver] [info] [Task Finished] Transform to mem in 0.253951s (14170 normal rows, 0 exceptions)
[2021-03-31 18:26:29.013] [driver] [info] [Task Finished] Transform to mem in 0.252651s (14002 normal rows, 0 exceptions)
[2021-03-31 18:26:29.266] [driver] [info] [Task Finished] Transform to mem in 0.253420s (14226 normal rows, 0 exceptions)
[2021-03-31 18:26:29.519] [driver] [info] [Task Finished] Transform to mem in 0.252839s (14206 normal rows, 0 exceptions)
[2021-03-31 18:26:29.772] [driver] [info] [Task Finished] Transform to mem in 0.252577s (13970 normal rows, 0 exceptions)
[2021-03-31 18:26:30.025] [driver] [info] [Task Finished] Transform to mem in 0.253309s (14237 normal rows, 0 exceptions)
[2021-03-31 18:26:30.278] [driver] [info] [Task Finished] Transform to mem in 0.252602s (14233 normal rows, 0 exceptions)
[2021-03-31 18:26:30.531] [driver] [info] [Task Finished] Transform to mem in 0.253819s (13920 normal rows, 0 exceptions)
[2021-03-31 18:26:30.786] [driver] [info] [Task Finished] Transform to mem in 0.254695s (14267 normal rows, 0 exceptions)
[2021-03-31 18:26:31.042] [driver] [info] [Task Finished] Transform to mem in 0.256060s (14248 normal rows, 0 exceptions)
[2021-03-31 18:26:31.295] [driver] [info] [Task Finished] Transform to mem in 0.252298s (13912 normal rows, 0 exceptions)
[2021-03-31 18:26:31.548] [driver] [info] [Task Finished] Transform to mem in 0.253207s (14248 normal rows, 0 exceptions)
[2021-03-31 18:26:31.801] [driver] [info] [Task Finished] Transform to mem in 0.253027s (14237 normal rows, 0 exceptions)
[2021-03-31 18:26:32.053] [driver] [info] [Task Finished] Transform to mem in 0.252577s (13988 normal rows, 0 exceptions)
[2021-03-31 18:26:32.306] [driver] [info] [Task Finished] Transform to mem in 0.252906s (14171 normal rows, 0 exceptions)
[2021-03-31 18:26:32.559] [driver] [info] [Task Finished] Transform to mem in 0.252716s (14234 normal rows, 0 exceptions)
[2021-03-31 18:26:32.813] [driver] [info] [Task Finished] Transform to mem in 0.253736s (14015 normal rows, 0 exceptions)
[2021-03-31 18:26:33.066] [driver] [info] [Task Finished] Transform to mem in 0.252931s (14176 normal rows, 0 exceptions)
[2021-03-31 18:26:33.319] [driver] [info] [Task Finished] Transform to mem in 0.252846s (14246 normal rows, 0 exceptions)
[2021-03-31 18:26:33.571] [driver] [info] [Task Finished] Transform to mem in 0.252554s (14059 normal rows, 0 exceptions)
[2021-03-31 18:26:33.824] [driver] [info] [Task Finished] Transform to mem in 0.252839s (14114 normal rows, 0 exceptions)
[2021-03-31 18:26:34.076] [driver] [info] [Task Finished] Transform to mem in 0.252381s (14238 normal rows, 0 exceptions)
[2021-03-31 18:26:34.332] [driver] [info] [Task Finished] Transform to mem in 0.255773s (14081 normal rows, 0 exceptions)
[2021-03-31 18:26:34.588] [driver] [info] [Task Finished] Transform to mem in 0.256029s (14095 normal rows, 0 exceptions)
[2021-03-31 18:26:34.841] [driver] [info] [Task Finished] Transform to mem in 0.253098s (14221 normal rows, 0 exceptions)
[2021-03-31 18:26:35.094] [driver] [info] [Task Finished] Transform to mem in 0.252959s (14086 normal rows, 0 exceptions)
[2021-03-31 18:26:35.347] [driver] [info] [Task Finished] Transform to mem in 0.253097s (14091 normal rows, 0 exceptions)
[2021-03-31 18:26:35.600] [driver] [info] [Task Finished] Transform to mem in 0.252980s (14218 normal rows, 0 exceptions)
[2021-03-31 18:26:35.853] [driver] [info] [Task Finished] Transform to mem in 0.252634s (14095 normal rows, 0 exceptions)
[2021-03-31 18:26:36.106] [driver] [info] [Task Finished] Transform to mem in 0.252502s (14113 normal rows, 0 exceptions)
[2021-03-31 18:26:36.358] [driver] [info] [Task Finished] Transform to mem in 0.252872s (14197 normal rows, 0 exceptions)
[2021-03-31 18:26:36.611] [driver] [info] [Task Finished] Transform to mem in 0.252810s (14114 normal rows, 0 exceptions)
[2021-03-31 18:26:36.864] [driver] [info] [Task Finished] Transform to mem in 0.252577s (14094 normal rows, 0 exceptions)
[2021-03-31 18:26:37.117] [driver] [info] [Task Finished] Transform to mem in 0.252782s (14197 normal rows, 0 exceptions)
[2021-03-31 18:26:37.369] [driver] [info] [Task Finished] Transform to mem in 0.252588s (14115 normal rows, 0 exceptions)
[2021-03-31 18:26:37.622] [driver] [info] [Task Finished] Transform to mem in 0.252597s (14083 normal rows, 0 exceptions)
[2021-03-31 18:26:37.875] [driver] [info] [Task Finished] Transform to mem in 0.253174s (14235 normal rows, 0 exceptions)
[2021-03-31 18:26:38.130] [driver] [info] [Task Finished] Transform to mem in 0.255306s (14144 normal rows, 0 exceptions)
[2021-03-31 18:26:38.386] [driver] [info] [Task Finished] Transform to mem in 0.255801s (14015 normal rows, 0 exceptions)
[2021-03-31 18:26:38.639] [driver] [info] [Task Finished] Transform to mem in 0.252875s (14237 normal rows, 0 exceptions)
[2021-03-31 18:26:38.892] [driver] [info] [Task Finished] Transform to mem in 0.252746s (14159 normal rows, 0 exceptions)
[2021-03-31 18:26:39.148] [driver] [info] [Task Finished] Transform to mem in 0.255707s (14010 normal rows, 0 exceptions)
[2021-03-31 18:26:39.401] [driver] [info] [Task Finished] Transform to mem in 0.253278s (14246 normal rows, 0 exceptions)
[2021-03-31 18:26:39.653] [driver] [info] [Task Finished] Transform to mem in 0.252600s (14178 normal rows, 0 exceptions)
[2021-03-31 18:26:39.906] [driver] [info] [Task Finished] Transform to mem in 0.252310s (13970 normal rows, 0 exceptions)
[2021-03-31 18:26:40.159] [driver] [info] [Task Finished] Transform to mem in 0.253013s (14253 normal rows, 0 exceptions)
[2021-03-31 18:26:40.412] [driver] [info] [Task Finished] Transform to mem in 0.252941s (14236 normal rows, 0 exceptions)
[2021-03-31 18:26:40.664] [driver] [info] [Task Finished] Transform to mem in 0.252006s (13901 normal rows, 0 exceptions)
[2021-03-31 18:26:40.917] [driver] [info] [Task Finished] Transform to mem in 0.253240s (14267 normal rows, 0 exceptions)
[2021-03-31 18:26:41.170] [driver] [info] [Task Finished] Transform to mem in 0.252703s (14248 normal rows, 0 exceptions)
[2021-03-31 18:26:41.422] [driver] [info] [Task Finished] Transform to mem in 0.252013s (13953 normal rows, 0 exceptions)
[2021-03-31 18:26:41.678] [driver] [info] [Task Finished] Transform to mem in 0.256142s (14207 normal rows, 0 exceptions)
[2021-03-31 18:26:41.934] [driver] [info] [Task Finished] Transform to mem in 0.256009s (14245 normal rows, 0 exceptions)
[2021-03-31 18:26:42.186] [driver] [info] [Task Finished] Transform to mem in 0.252187s (13999 normal rows, 0 exceptions)
[2021-03-31 18:26:42.439] [driver] [info] [Task Finished] Transform to mem in 0.252815s (14158 normal rows, 0 exceptions)
[2021-03-31 18:26:42.695] [driver] [info] [Task Finished] Transform to mem in 0.256091s (14242 normal rows, 0 exceptions)
[2021-03-31 18:26:42.947] [driver] [info] [Task Finished] Transform to mem in 0.252349s (14020 normal rows, 0 exceptions)
[2021-03-31 18:26:43.200] [driver] [info] [Task Finished] Transform to mem in 0.252970s (14173 normal rows, 0 exceptions)
[2021-03-31 18:26:43.453] [driver] [info] [Task Finished] Transform to mem in 0.252799s (14235 normal rows, 0 exceptions)
[2021-03-31 18:26:43.706] [driver] [info] [Task Finished] Transform to mem in 0.252397s (14054 normal rows, 0 exceptions)
[2021-03-31 18:26:43.958] [driver] [info] [Task Finished] Transform to mem in 0.252787s (14114 normal rows, 0 exceptions)
[2021-03-31 18:26:44.211] [driver] [info] [Task Finished] Transform to mem in 0.252916s (14238 normal rows, 0 exceptions)
[2021-03-31 18:26:44.464] [driver] [info] [Task Finished] Transform to mem in 0.252478s (14081 normal rows, 0 exceptions)
[2021-03-31 18:26:44.717] [driver] [info] [Task Finished] Transform to mem in 0.252743s (14095 normal rows, 0 exceptions)
[2021-03-31 18:26:44.969] [driver] [info] [Task Finished] Transform to mem in 0.252839s (14221 normal rows, 0 exceptions)
[2021-03-31 18:26:45.222] [driver] [info] [Task Finished] Transform to mem in 0.252544s (14089 normal rows, 0 exceptions)
[2021-03-31 18:26:45.478] [driver] [info] [Task Finished] Transform to mem in 0.255771s (14094 normal rows, 0 exceptions)
[2021-03-31 18:26:45.734] [driver] [info] [Task Finished] Transform to mem in 0.256090s (14212 normal rows, 0 exceptions)
[2021-03-31 18:26:45.987] [driver] [info] [Task Finished] Transform to mem in 0.252661s (14108 normal rows, 0 exceptions)
[2021-03-31 18:26:46.239] [driver] [info] [Task Finished] Transform to mem in 0.252650s (14100 normal rows, 0 exceptions)
[2021-03-31 18:26:46.496] [driver] [info] [Task Finished] Transform to mem in 0.256544s (14217 normal rows, 0 exceptions)
[2021-03-31 18:26:46.748] [driver] [info] [Task Finished] Transform to mem in 0.252282s (14094 normal rows, 0 exceptions)
[2021-03-31 18:26:47.001] [driver] [info] [Task Finished] Transform to mem in 0.252863s (14094 normal rows, 0 exceptions)
[2021-03-31 18:26:47.255] [driver] [info] [Task Finished] Transform to mem in 0.253754s (14223 normal rows, 0 exceptions)
[2021-03-31 18:26:47.507] [driver] [info] [Task Finished] Transform to mem in 0.252467s (14089 normal rows, 0 exceptions)
[2021-03-31 18:26:47.760] [driver] [info] [Task Finished] Transform to mem in 0.252533s (14086 normal rows, 0 exceptions)
[2021-03-31 18:26:48.013] [driver] [info] [Task Finished] Transform to mem in 0.253097s (14232 normal rows, 0 exceptions)
[2021-03-31 18:26:48.266] [driver] [info] [Task Finished] Transform to mem in 0.252620s (14170 normal rows, 0 exceptions)
[2021-03-31 18:26:48.518] [driver] [info] [Task Finished] Transform to mem in 0.252114s (14002 normal rows, 0 exceptions)
[2021-03-31 18:26:48.771] [driver] [info] [Task Finished] Transform to mem in 0.253164s (14226 normal rows, 0 exceptions)
[2021-03-31 18:26:49.027] [driver] [info] [Task Finished] Transform to mem in 0.255907s (14206 normal rows, 0 exceptions)
[2021-03-31 18:26:49.282] [driver] [info] [Task Finished] Transform to mem in 0.255487s (13961 normal rows, 0 exceptions)
[2021-03-31 18:26:49.535] [driver] [info] [Task Finished] Transform to mem in 0.253105s (14246 normal rows, 0 exceptions)
[2021-03-31 18:26:49.788] [driver] [info] [Task Finished] Transform to mem in 0.252587s (14233 normal rows, 0 exceptions)
[2021-03-31 18:26:50.083] [driver] [info] [Task Finished] Transform to mem in 0.294500s (16320 normal rows, 0 exceptions)
[2021-03-31 18:26:50.083] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 40.5612s
[2021-03-31 18:26:50.083] [global] [info] [Transform Stage] Stage 0 total wall clock time: 40.5588s, 54108450 input rows, time to process 1 row via fast path: 0.000749583ms

[2021-03-31 18:26:50.120] [global] [info] Writing 2264570 rows as output to file (344.74 MB)
[2021-03-31 18:26:50.523] [driver] [info] [Task Finished] write to file in 0.403720s (2264570 rows, 344.74 MB)
[2021-03-31 18:26:50.523] [global] [info] writing output took 0.440809s
[2021-03-31 18:26:50.523] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.440897s
[2021-03-31 18:26:50.523] [global] [info] [Transform Stage] Stage 0 took 41.2061s
[2021-03-31 18:26:50.523] [global] [info] Query Execution took 41.2625s. (planning: 0.0564288s, execution: 41.2061s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 31 2021, 05:47:10) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z2_preprocessed/zillow_Z2_10G.csv']
Tuplex startup time: 0.004991292953491211
Tuplex job time: 41.531091928482056 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 0, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '32G', 'tuplex.env.hostname': '6bcf8f6a24af', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '8MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.0195859,"llvm_optimization_time_s":0.135517,"llvm_compilation_time_s":0.0610464,"total_compilation_time_s":0.20395,"sampling_time_s":0.114391,"stages":[{"stage_no":0,"fast_path_wall_time_s":40.5588,"fast_path_time_s":40.5612,"fast_path_per_row_time_ns":749.583,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.004991292953491211, "jobTime": 41.531091928482056}
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
