[2021-03-31 18:25:26.555] [global] [error] could not find backend, defaulting to local
[2021-03-31 18:25:26.556] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-31 18:25:26.556] [local ee] [info] initializing LLVM backend
[2021-03-31 18:25:26.556] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-31 18:25:26.556] [LLVM] [info] compiling code for skylake-avx512
[2021-03-31 18:25:26.557] [memory] [info] allocated bitmap managed memory region (32.00 GB, 32.00 MB block size)
[2021-03-31 18:25:26.557] [local execution engine] [info] started driver (32.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-31 18:25:26.557] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2021-03-31 18:25:26.557] [global] [info] sampled file:///data/zillow/Z2_preprocessed/zillow_Z2_10G.csv on 256.00 KB
[2021-03-31 18:25:26.696] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:25:26.712] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:25:26.720] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:25:26.746] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:25:26.765] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:25:26.778] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:25:26.809] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:25:26.825] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:25:26.827] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:25:26.830] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:25:26.832] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:25:26.847] [logical planner] [info] logical optimization took 0.019619ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-31 18:25:26.848] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (13 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:25:26.848] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-31 18:25:26.849] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-31 18:25:26.849] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-31 18:25:26.850] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-31 18:25:26.851] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-31 18:25:26.851] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-31 18:25:26.852] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> f64
[2021-03-31 18:25:26.853] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64)) -> i64
[2021-03-31 18:25:26.854] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64)) -> str
[2021-03-31 18:25:26.855] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64,str)) -> i64
[2021-03-31 18:25:26.856] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:25:26.856] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:25:26.857] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:25:26.882] [global] [info] Stage0 depends on: 
[2021-03-31 18:25:27.026] [global] [info] Optimization via LLVM passes took 0.136162 ms
[2021-03-31 18:25:27.087] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2021-03-31 18:25:27.087] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.205108s
[2021-03-31 18:25:27.088] [local ee] [info] split /data/zillow/Z2_preprocessed/zillow_Z2_10G.csv into 160 tasks
[2021-03-31 18:25:27.346] [driver] [info] [Task Finished] Transform to mem in 0.258479s (14226 normal rows, 0 exceptions)
[2021-03-31 18:25:27.603] [driver] [info] [Task Finished] Transform to mem in 0.257314s (14206 normal rows, 0 exceptions)
[2021-03-31 18:25:27.862] [driver] [info] [Task Finished] Transform to mem in 0.258310s (13970 normal rows, 0 exceptions)
[2021-03-31 18:25:28.121] [driver] [info] [Task Finished] Transform to mem in 0.259261s (14253 normal rows, 0 exceptions)
[2021-03-31 18:25:28.380] [driver] [info] [Task Finished] Transform to mem in 0.258732s (14217 normal rows, 0 exceptions)
[2021-03-31 18:25:28.635] [driver] [info] [Task Finished] Transform to mem in 0.254890s (13920 normal rows, 0 exceptions)
[2021-03-31 18:25:28.891] [driver] [info] [Task Finished] Transform to mem in 0.256524s (14267 normal rows, 0 exceptions)
[2021-03-31 18:25:29.147] [driver] [info] [Task Finished] Transform to mem in 0.255606s (14248 normal rows, 0 exceptions)
[2021-03-31 18:25:29.402] [driver] [info] [Task Finished] Transform to mem in 0.255518s (13953 normal rows, 0 exceptions)
[2021-03-31 18:25:29.659] [driver] [info] [Task Finished] Transform to mem in 0.256297s (14207 normal rows, 0 exceptions)
[2021-03-31 18:25:29.915] [driver] [info] [Task Finished] Transform to mem in 0.255852s (14237 normal rows, 0 exceptions)
[2021-03-31 18:25:30.170] [driver] [info] [Task Finished] Transform to mem in 0.255263s (13988 normal rows, 0 exceptions)
[2021-03-31 18:25:30.426] [driver] [info] [Task Finished] Transform to mem in 0.256262s (14177 normal rows, 0 exceptions)
[2021-03-31 18:25:30.682] [driver] [info] [Task Finished] Transform to mem in 0.255898s (14242 normal rows, 0 exceptions)
[2021-03-31 18:25:30.937] [driver] [info] [Task Finished] Transform to mem in 0.255164s (14020 normal rows, 0 exceptions)
[2021-03-31 18:25:31.193] [driver] [info] [Task Finished] Transform to mem in 0.255747s (14162 normal rows, 0 exceptions)
[2021-03-31 18:25:31.448] [driver] [info] [Task Finished] Transform to mem in 0.255373s (14241 normal rows, 0 exceptions)
[2021-03-31 18:25:31.707] [driver] [info] [Task Finished] Transform to mem in 0.258601s (14059 normal rows, 0 exceptions)
[2021-03-31 18:25:31.963] [driver] [info] [Task Finished] Transform to mem in 0.255542s (14114 normal rows, 0 exceptions)
[2021-03-31 18:25:32.221] [driver] [info] [Task Finished] Transform to mem in 0.258628s (14238 normal rows, 0 exceptions)
[2021-03-31 18:25:32.477] [driver] [info] [Task Finished] Transform to mem in 0.255509s (14081 normal rows, 0 exceptions)
[2021-03-31 18:25:32.733] [driver] [info] [Task Finished] Transform to mem in 0.255920s (14095 normal rows, 0 exceptions)
[2021-03-31 18:25:32.989] [driver] [info] [Task Finished] Transform to mem in 0.255909s (14221 normal rows, 0 exceptions)
[2021-03-31 18:25:33.244] [driver] [info] [Task Finished] Transform to mem in 0.255621s (14086 normal rows, 0 exceptions)
[2021-03-31 18:25:33.500] [driver] [info] [Task Finished] Transform to mem in 0.255423s (14097 normal rows, 0 exceptions)
[2021-03-31 18:25:33.756] [driver] [info] [Task Finished] Transform to mem in 0.255836s (14212 normal rows, 0 exceptions)
[2021-03-31 18:25:34.012] [driver] [info] [Task Finished] Transform to mem in 0.256343s (14108 normal rows, 0 exceptions)
[2021-03-31 18:25:34.267] [driver] [info] [Task Finished] Transform to mem in 0.255103s (14100 normal rows, 0 exceptions)
[2021-03-31 18:25:34.523] [driver] [info] [Task Finished] Transform to mem in 0.256087s (14212 normal rows, 0 exceptions)
[2021-03-31 18:25:34.779] [driver] [info] [Task Finished] Transform to mem in 0.255468s (14099 normal rows, 0 exceptions)
[2021-03-31 18:25:35.034] [driver] [info] [Task Finished] Transform to mem in 0.255681s (14094 normal rows, 0 exceptions)
[2021-03-31 18:25:35.294] [driver] [info] [Task Finished] Transform to mem in 0.259299s (14223 normal rows, 0 exceptions)
[2021-03-31 18:25:35.549] [driver] [info] [Task Finished] Transform to mem in 0.255367s (14089 normal rows, 0 exceptions)
[2021-03-31 18:25:35.808] [driver] [info] [Task Finished] Transform to mem in 0.259168s (14083 normal rows, 0 exceptions)
[2021-03-31 18:25:36.064] [driver] [info] [Task Finished] Transform to mem in 0.256280s (14235 normal rows, 0 exceptions)
[2021-03-31 18:25:36.320] [driver] [info] [Task Finished] Transform to mem in 0.255678s (14170 normal rows, 0 exceptions)
[2021-03-31 18:25:36.576] [driver] [info] [Task Finished] Transform to mem in 0.255681s (14002 normal rows, 0 exceptions)
[2021-03-31 18:25:36.832] [driver] [info] [Task Finished] Transform to mem in 0.256330s (14226 normal rows, 0 exceptions)
[2021-03-31 18:25:37.088] [driver] [info] [Task Finished] Transform to mem in 0.255686s (14206 normal rows, 0 exceptions)
[2021-03-31 18:25:37.345] [driver] [info] [Task Finished] Transform to mem in 0.256990s (13961 normal rows, 0 exceptions)
[2021-03-31 18:25:37.602] [driver] [info] [Task Finished] Transform to mem in 0.257575s (14246 normal rows, 0 exceptions)
[2021-03-31 18:25:37.860] [driver] [info] [Task Finished] Transform to mem in 0.257219s (14233 normal rows, 0 exceptions)
[2021-03-31 18:25:38.116] [driver] [info] [Task Finished] Transform to mem in 0.256707s (13915 normal rows, 0 exceptions)
[2021-03-31 18:25:38.374] [driver] [info] [Task Finished] Transform to mem in 0.257227s (14253 normal rows, 0 exceptions)
[2021-03-31 18:25:38.631] [driver] [info] [Task Finished] Transform to mem in 0.257300s (14267 normal rows, 0 exceptions)
[2021-03-31 18:25:38.888] [driver] [info] [Task Finished] Transform to mem in 0.257005s (13892 normal rows, 0 exceptions)
[2021-03-31 18:25:39.148] [driver] [info] [Task Finished] Transform to mem in 0.260313s (14245 normal rows, 0 exceptions)
[2021-03-31 18:25:39.409] [driver] [info] [Task Finished] Transform to mem in 0.260686s (14260 normal rows, 0 exceptions)
[2021-03-31 18:25:39.666] [driver] [info] [Task Finished] Transform to mem in 0.256792s (13978 normal rows, 0 exceptions)
[2021-03-31 18:25:39.923] [driver] [info] [Task Finished] Transform to mem in 0.257407s (14181 normal rows, 0 exceptions)
[2021-03-31 18:25:40.180] [driver] [info] [Task Finished] Transform to mem in 0.256577s (14234 normal rows, 0 exceptions)
[2021-03-31 18:25:40.436] [driver] [info] [Task Finished] Transform to mem in 0.256576s (14015 normal rows, 0 exceptions)
[2021-03-31 18:25:40.694] [driver] [info] [Task Finished] Transform to mem in 0.257690s (14142 normal rows, 0 exceptions)
[2021-03-31 18:25:40.952] [driver] [info] [Task Finished] Transform to mem in 0.257549s (14280 normal rows, 0 exceptions)
[2021-03-31 18:25:41.209] [driver] [info] [Task Finished] Transform to mem in 0.257285s (13982 normal rows, 0 exceptions)
[2021-03-31 18:25:41.467] [driver] [info] [Task Finished] Transform to mem in 0.257627s (14173 normal rows, 0 exceptions)
[2021-03-31 18:25:41.723] [driver] [info] [Task Finished] Transform to mem in 0.256725s (14235 normal rows, 0 exceptions)
[2021-03-31 18:25:41.979] [driver] [info] [Task Finished] Transform to mem in 0.255619s (14054 normal rows, 0 exceptions)
[2021-03-31 18:25:42.233] [driver] [info] [Task Finished] Transform to mem in 0.254427s (14129 normal rows, 0 exceptions)
[2021-03-31 18:25:42.488] [driver] [info] [Task Finished] Transform to mem in 0.254029s (14223 normal rows, 0 exceptions)
[2021-03-31 18:25:42.745] [driver] [info] [Task Finished] Transform to mem in 0.257596s (14081 normal rows, 0 exceptions)
[2021-03-31 18:25:43.001] [driver] [info] [Task Finished] Transform to mem in 0.255474s (14108 normal rows, 0 exceptions)
[2021-03-31 18:25:43.260] [driver] [info] [Task Finished] Transform to mem in 0.259209s (14208 normal rows, 0 exceptions)
[2021-03-31 18:25:43.514] [driver] [info] [Task Finished] Transform to mem in 0.253676s (14089 normal rows, 0 exceptions)
[2021-03-31 18:25:43.769] [driver] [info] [Task Finished] Transform to mem in 0.255301s (14094 normal rows, 0 exceptions)
[2021-03-31 18:25:44.025] [driver] [info] [Task Finished] Transform to mem in 0.256042s (14232 normal rows, 0 exceptions)
[2021-03-31 18:25:44.281] [driver] [info] [Task Finished] Transform to mem in 0.255780s (14088 normal rows, 0 exceptions)
[2021-03-31 18:25:44.537] [driver] [info] [Task Finished] Transform to mem in 0.256291s (14106 normal rows, 0 exceptions)
[2021-03-31 18:25:44.792] [driver] [info] [Task Finished] Transform to mem in 0.254754s (14211 normal rows, 0 exceptions)
[2021-03-31 18:25:45.047] [driver] [info] [Task Finished] Transform to mem in 0.255367s (14113 normal rows, 0 exceptions)
[2021-03-31 18:25:45.303] [driver] [info] [Task Finished] Transform to mem in 0.255918s (14075 normal rows, 0 exceptions)
[2021-03-31 18:25:45.559] [driver] [info] [Task Finished] Transform to mem in 0.256271s (14223 normal rows, 0 exceptions)
[2021-03-31 18:25:45.814] [driver] [info] [Task Finished] Transform to mem in 0.255007s (14095 normal rows, 0 exceptions)
[2021-03-31 18:25:46.068] [driver] [info] [Task Finished] Transform to mem in 0.253614s (14086 normal rows, 0 exceptions)
[2021-03-31 18:25:46.326] [driver] [info] [Task Finished] Transform to mem in 0.257723s (14226 normal rows, 0 exceptions)
[2021-03-31 18:25:46.580] [driver] [info] [Task Finished] Transform to mem in 0.254349s (14170 normal rows, 0 exceptions)
[2021-03-31 18:25:46.837] [driver] [info] [Task Finished] Transform to mem in 0.257209s (14002 normal rows, 0 exceptions)
[2021-03-31 18:25:47.093] [driver] [info] [Task Finished] Transform to mem in 0.255775s (14226 normal rows, 0 exceptions)
[2021-03-31 18:25:47.347] [driver] [info] [Task Finished] Transform to mem in 0.254032s (14206 normal rows, 0 exceptions)
[2021-03-31 18:25:47.602] [driver] [info] [Task Finished] Transform to mem in 0.254816s (13970 normal rows, 0 exceptions)
[2021-03-31 18:25:47.858] [driver] [info] [Task Finished] Transform to mem in 0.256247s (14237 normal rows, 0 exceptions)
[2021-03-31 18:25:48.116] [driver] [info] [Task Finished] Transform to mem in 0.257602s (14233 normal rows, 0 exceptions)
[2021-03-31 18:25:48.372] [driver] [info] [Task Finished] Transform to mem in 0.255785s (13920 normal rows, 0 exceptions)
[2021-03-31 18:25:48.628] [driver] [info] [Task Finished] Transform to mem in 0.256552s (14267 normal rows, 0 exceptions)
[2021-03-31 18:25:48.885] [driver] [info] [Task Finished] Transform to mem in 0.256338s (14248 normal rows, 0 exceptions)
[2021-03-31 18:25:49.140] [driver] [info] [Task Finished] Transform to mem in 0.255057s (13912 normal rows, 0 exceptions)
[2021-03-31 18:25:49.396] [driver] [info] [Task Finished] Transform to mem in 0.256015s (14248 normal rows, 0 exceptions)
[2021-03-31 18:25:49.652] [driver] [info] [Task Finished] Transform to mem in 0.255872s (14237 normal rows, 0 exceptions)
[2021-03-31 18:25:49.907] [driver] [info] [Task Finished] Transform to mem in 0.255113s (13988 normal rows, 0 exceptions)
[2021-03-31 18:25:50.164] [driver] [info] [Task Finished] Transform to mem in 0.256833s (14171 normal rows, 0 exceptions)
[2021-03-31 18:25:50.421] [driver] [info] [Task Finished] Transform to mem in 0.257745s (14234 normal rows, 0 exceptions)
[2021-03-31 18:25:50.675] [driver] [info] [Task Finished] Transform to mem in 0.253581s (14015 normal rows, 0 exceptions)
[2021-03-31 18:25:50.931] [driver] [info] [Task Finished] Transform to mem in 0.256059s (14176 normal rows, 0 exceptions)
[2021-03-31 18:25:51.188] [driver] [info] [Task Finished] Transform to mem in 0.256760s (14246 normal rows, 0 exceptions)
[2021-03-31 18:25:51.444] [driver] [info] [Task Finished] Transform to mem in 0.256672s (14059 normal rows, 0 exceptions)
[2021-03-31 18:25:51.701] [driver] [info] [Task Finished] Transform to mem in 0.256477s (14114 normal rows, 0 exceptions)
[2021-03-31 18:25:51.958] [driver] [info] [Task Finished] Transform to mem in 0.256804s (14238 normal rows, 0 exceptions)
[2021-03-31 18:25:52.215] [driver] [info] [Task Finished] Transform to mem in 0.256851s (14081 normal rows, 0 exceptions)
[2021-03-31 18:25:52.471] [driver] [info] [Task Finished] Transform to mem in 0.256214s (14095 normal rows, 0 exceptions)
[2021-03-31 18:25:52.727] [driver] [info] [Task Finished] Transform to mem in 0.256535s (14221 normal rows, 0 exceptions)
[2021-03-31 18:25:52.984] [driver] [info] [Task Finished] Transform to mem in 0.256233s (14086 normal rows, 0 exceptions)
[2021-03-31 18:25:53.240] [driver] [info] [Task Finished] Transform to mem in 0.256525s (14091 normal rows, 0 exceptions)
[2021-03-31 18:25:53.497] [driver] [info] [Task Finished] Transform to mem in 0.256792s (14218 normal rows, 0 exceptions)
[2021-03-31 18:25:53.756] [driver] [info] [Task Finished] Transform to mem in 0.259367s (14095 normal rows, 0 exceptions)
[2021-03-31 18:25:54.012] [driver] [info] [Task Finished] Transform to mem in 0.256127s (14113 normal rows, 0 exceptions)
[2021-03-31 18:25:54.273] [driver] [info] [Task Finished] Transform to mem in 0.260021s (14197 normal rows, 0 exceptions)
[2021-03-31 18:25:54.529] [driver] [info] [Task Finished] Transform to mem in 0.256580s (14114 normal rows, 0 exceptions)
[2021-03-31 18:25:54.786] [driver] [info] [Task Finished] Transform to mem in 0.257248s (14094 normal rows, 0 exceptions)
[2021-03-31 18:25:55.044] [driver] [info] [Task Finished] Transform to mem in 0.257154s (14197 normal rows, 0 exceptions)
[2021-03-31 18:25:55.300] [driver] [info] [Task Finished] Transform to mem in 0.256108s (14115 normal rows, 0 exceptions)
[2021-03-31 18:25:55.557] [driver] [info] [Task Finished] Transform to mem in 0.257268s (14083 normal rows, 0 exceptions)
[2021-03-31 18:25:55.815] [driver] [info] [Task Finished] Transform to mem in 0.257596s (14235 normal rows, 0 exceptions)
[2021-03-31 18:25:56.071] [driver] [info] [Task Finished] Transform to mem in 0.256450s (14144 normal rows, 0 exceptions)
[2021-03-31 18:25:56.327] [driver] [info] [Task Finished] Transform to mem in 0.256265s (14015 normal rows, 0 exceptions)
[2021-03-31 18:25:56.585] [driver] [info] [Task Finished] Transform to mem in 0.257577s (14237 normal rows, 0 exceptions)
[2021-03-31 18:25:56.840] [driver] [info] [Task Finished] Transform to mem in 0.254904s (14159 normal rows, 0 exceptions)
[2021-03-31 18:25:57.094] [driver] [info] [Task Finished] Transform to mem in 0.254549s (14010 normal rows, 0 exceptions)
[2021-03-31 18:25:57.353] [driver] [info] [Task Finished] Transform to mem in 0.258812s (14246 normal rows, 0 exceptions)
[2021-03-31 18:25:57.608] [driver] [info] [Task Finished] Transform to mem in 0.254921s (14178 normal rows, 0 exceptions)
[2021-03-31 18:25:57.866] [driver] [info] [Task Finished] Transform to mem in 0.257477s (13970 normal rows, 0 exceptions)
[2021-03-31 18:25:58.121] [driver] [info] [Task Finished] Transform to mem in 0.255751s (14253 normal rows, 0 exceptions)
[2021-03-31 18:25:58.376] [driver] [info] [Task Finished] Transform to mem in 0.254952s (14236 normal rows, 0 exceptions)
[2021-03-31 18:25:58.630] [driver] [info] [Task Finished] Transform to mem in 0.254050s (13901 normal rows, 0 exceptions)
[2021-03-31 18:25:58.886] [driver] [info] [Task Finished] Transform to mem in 0.256029s (14267 normal rows, 0 exceptions)
[2021-03-31 18:25:59.142] [driver] [info] [Task Finished] Transform to mem in 0.255496s (14248 normal rows, 0 exceptions)
[2021-03-31 18:25:59.397] [driver] [info] [Task Finished] Transform to mem in 0.255217s (13953 normal rows, 0 exceptions)
[2021-03-31 18:25:59.654] [driver] [info] [Task Finished] Transform to mem in 0.256573s (14207 normal rows, 0 exceptions)
[2021-03-31 18:25:59.910] [driver] [info] [Task Finished] Transform to mem in 0.255724s (14245 normal rows, 0 exceptions)
[2021-03-31 18:26:00.164] [driver] [info] [Task Finished] Transform to mem in 0.254917s (13999 normal rows, 0 exceptions)
[2021-03-31 18:26:00.419] [driver] [info] [Task Finished] Transform to mem in 0.254334s (14158 normal rows, 0 exceptions)
[2021-03-31 18:26:00.673] [driver] [info] [Task Finished] Transform to mem in 0.254548s (14242 normal rows, 0 exceptions)
[2021-03-31 18:26:00.928] [driver] [info] [Task Finished] Transform to mem in 0.254486s (14020 normal rows, 0 exceptions)
[2021-03-31 18:26:01.188] [driver] [info] [Task Finished] Transform to mem in 0.259659s (14173 normal rows, 0 exceptions)
[2021-03-31 18:26:01.447] [driver] [info] [Task Finished] Transform to mem in 0.259389s (14235 normal rows, 0 exceptions)
[2021-03-31 18:26:01.704] [driver] [info] [Task Finished] Transform to mem in 0.256756s (14054 normal rows, 0 exceptions)
[2021-03-31 18:26:01.961] [driver] [info] [Task Finished] Transform to mem in 0.257344s (14114 normal rows, 0 exceptions)
[2021-03-31 18:26:02.217] [driver] [info] [Task Finished] Transform to mem in 0.256254s (14238 normal rows, 0 exceptions)
[2021-03-31 18:26:02.473] [driver] [info] [Task Finished] Transform to mem in 0.255907s (14081 normal rows, 0 exceptions)
[2021-03-31 18:26:02.729] [driver] [info] [Task Finished] Transform to mem in 0.255963s (14095 normal rows, 0 exceptions)
[2021-03-31 18:26:02.985] [driver] [info] [Task Finished] Transform to mem in 0.256161s (14221 normal rows, 0 exceptions)
[2021-03-31 18:26:03.242] [driver] [info] [Task Finished] Transform to mem in 0.256996s (14089 normal rows, 0 exceptions)
[2021-03-31 18:26:03.500] [driver] [info] [Task Finished] Transform to mem in 0.257301s (14094 normal rows, 0 exceptions)
[2021-03-31 18:26:03.755] [driver] [info] [Task Finished] Transform to mem in 0.255586s (14212 normal rows, 0 exceptions)
[2021-03-31 18:26:04.011] [driver] [info] [Task Finished] Transform to mem in 0.255835s (14108 normal rows, 0 exceptions)
[2021-03-31 18:26:04.267] [driver] [info] [Task Finished] Transform to mem in 0.255711s (14100 normal rows, 0 exceptions)
[2021-03-31 18:26:04.523] [driver] [info] [Task Finished] Transform to mem in 0.256338s (14217 normal rows, 0 exceptions)
[2021-03-31 18:26:04.781] [driver] [info] [Task Finished] Transform to mem in 0.258119s (14094 normal rows, 0 exceptions)
[2021-03-31 18:26:05.036] [driver] [info] [Task Finished] Transform to mem in 0.254630s (14094 normal rows, 0 exceptions)
[2021-03-31 18:26:05.294] [driver] [info] [Task Finished] Transform to mem in 0.258141s (14223 normal rows, 0 exceptions)
[2021-03-31 18:26:05.550] [driver] [info] [Task Finished] Transform to mem in 0.255637s (14089 normal rows, 0 exceptions)
[2021-03-31 18:26:05.806] [driver] [info] [Task Finished] Transform to mem in 0.256330s (14086 normal rows, 0 exceptions)
[2021-03-31 18:26:06.063] [driver] [info] [Task Finished] Transform to mem in 0.257064s (14232 normal rows, 0 exceptions)
[2021-03-31 18:26:06.320] [driver] [info] [Task Finished] Transform to mem in 0.256218s (14170 normal rows, 0 exceptions)
[2021-03-31 18:26:06.574] [driver] [info] [Task Finished] Transform to mem in 0.254461s (14002 normal rows, 0 exceptions)
[2021-03-31 18:26:06.829] [driver] [info] [Task Finished] Transform to mem in 0.255250s (14226 normal rows, 0 exceptions)
[2021-03-31 18:26:07.084] [driver] [info] [Task Finished] Transform to mem in 0.254319s (14206 normal rows, 0 exceptions)
[2021-03-31 18:26:07.337] [driver] [info] [Task Finished] Transform to mem in 0.253740s (13961 normal rows, 0 exceptions)
[2021-03-31 18:26:07.592] [driver] [info] [Task Finished] Transform to mem in 0.254912s (14246 normal rows, 0 exceptions)
[2021-03-31 18:26:07.847] [driver] [info] [Task Finished] Transform to mem in 0.254805s (14233 normal rows, 0 exceptions)
[2021-03-31 18:26:08.143] [driver] [info] [Task Finished] Transform to mem in 0.296115s (16320 normal rows, 0 exceptions)
[2021-03-31 18:26:08.143] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 41.0559s
[2021-03-31 18:26:08.143] [global] [info] [Transform Stage] Stage 0 total wall clock time: 41.0533s, 54108450 input rows, time to process 1 row via fast path: 0.000758722ms

[2021-03-31 18:26:08.181] [global] [info] Writing 2264570 rows as output to file (344.74 MB)
[2021-03-31 18:26:08.614] [driver] [info] [Task Finished] write to file in 0.433792s (2264570 rows, 344.74 MB)
[2021-03-31 18:26:08.614] [global] [info] writing output took 0.471084s
[2021-03-31 18:26:08.615] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.471173s
[2021-03-31 18:26:08.615] [global] [info] [Transform Stage] Stage 0 took 41.7322s
[2021-03-31 18:26:08.615] [global] [info] Query Execution took 41.7891s. (planning: 0.0568319s, execution: 41.7323s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 31 2021, 05:47:10) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z2_preprocessed/zillow_Z2_10G.csv']
Tuplex startup time: 0.004980325698852539
Tuplex job time: 42.05902361869812 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 0, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '32G', 'tuplex.env.hostname': '6bcf8f6a24af', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '8MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.0196186,"llvm_optimization_time_s":0.136162,"llvm_compilation_time_s":0.0615513,"total_compilation_time_s":0.205106,"sampling_time_s":0.115898,"stages":[{"stage_no":0,"fast_path_wall_time_s":41.0533,"fast_path_time_s":41.0559,"fast_path_per_row_time_ns":758.722,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.004980325698852539, "jobTime": 42.05902361869812}
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
