[2021-03-25 10:30:10.304] [global] [error] could not find backend, defaulting to local
[2021-03-25 10:30:10.305] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-25 10:30:10.305] [local ee] [info] initializing LLVM backend
[2021-03-25 10:30:10.305] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-25 10:30:10.305] [LLVM] [info] compiling code for skylake-avx512
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/1 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.306] [E/1] [info] starting detached process queue
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/2 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.306] [E/2] [info] starting detached process queue
[2021-03-25 10:30:10.306] [E/1] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/3 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.306] [E/2] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.306] [E/3] [info] starting detached process queue
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/4 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.306] [E/4] [info] starting detached process queue
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/5 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.306] [E/3] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.306] [E/5] [info] starting detached process queue
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/6 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.306] [E/4] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.306] [E/6] [info] starting detached process queue
[2021-03-25 10:30:10.306] [E/5] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/7 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.306] [E/7] [info] starting detached process queue
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/8 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.306] [E/6] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.306] [E/7] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.306] [E/8] [info] starting detached process queue
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/9 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.306] [E/9] [info] starting detached process queue
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/10 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.306] [E/8] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/11 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.306] [E/10] [info] starting detached process queue
[2021-03-25 10:30:10.306] [E/9] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.306] [E/11] [info] starting detached process queue
[2021-03-25 10:30:10.306] [E/10] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/12 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.306] [E/11] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.306] [E/12] [info] starting detached process queue
[2021-03-25 10:30:10.306] [local execution engine] [info] started local executor E/13 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.307] [local execution engine] [info] started local executor E/14 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.307] [E/13] [info] starting detached process queue
[2021-03-25 10:30:10.307] [E/12] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.307] [E/14] [info] starting detached process queue
[2021-03-25 10:30:10.307] [local execution engine] [info] started local executor E/15 (6.00 GB, 32.00 MB default partition size)
[2021-03-25 10:30:10.307] [E/15] [info] starting detached process queue
[2021-03-25 10:30:10.307] [E/13] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.307] [E/14] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.307] [E/15] [info] initialized runtime memory (4.00 MB)
[2021-03-25 10:30:10.307] [memory] [info] allocated bitmap managed memory region (10.00 GB, 32.00 MB block size)
[2021-03-25 10:30:10.307] [local execution engine] [info] started driver (10.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-25 10:30:10.307] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2021-03-25 10:30:10.307] [global] [info] sampled file:///data/zillow/Zdirty/zillow_dirty@10G.csv on 256.00 KB
[2021-03-25 10:30:10.450] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-25 10:30:10.468] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-25 10:30:10.481] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-25 10:30:10.515] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-25 10:30:10.546] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-25 10:30:10.567] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-25 10:30:10.606] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-25 10:30:10.622] [global] [info] performing static typing for UDF in operator map
[2021-03-25 10:30:10.626] [global] [info] performing static typing for UDF in operator map
[2021-03-25 10:30:10.630] [global] [info] performing static typing for UDF in operator map
[2021-03-25 10:30:10.633] [global] [info] performing static typing for UDF in operator map
[2021-03-25 10:30:10.636] [global] [error] unsupported node type
[2021-03-25 10:30:10.636] [global] [error] unsupported node type
[2021-03-25 10:30:10.636] [global] [error] unsupported node type
[2021-03-25 10:30:10.636] [global] [error] unsupported node type
[2021-03-25 10:30:10.660] [logical planner] [info] logical optimization took 0.033765ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-25 10:30:10.660] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (19 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],Option[i64],f64,i64,str,str,i64)
[2021-03-25 10:30:10.661] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> boolean
[2021-03-25 10:30:10.661] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> Option[i64]
[2021-03-25 10:30:10.662] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],Option[i64])) -> boolean
[2021-03-25 10:30:10.663] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],Option[i64])) -> str
[2021-03-25 10:30:10.664] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],Option[i64],str)) -> boolean
[2021-03-25 10:30:10.664] [codegen] [info] generating function extractZipcode for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],Option[i64],str)) -> str
[2021-03-25 10:30:10.665] [codegen] [info] generating lambda function for (str) -> str
[2021-03-25 10:30:10.665] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-25 10:30:10.666] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],Option[i64],str,str)) -> Option[f64]
[2021-03-25 10:30:10.667] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],Option[i64],str,str,Option[f64])) -> boolean
[2021-03-25 10:30:10.668] [codegen] [info] generating lambda function for (Option[f64]) -> f64
[2021-03-25 10:30:10.668] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],Option[i64],str,str,f64)) -> Option[i64]
[2021-03-25 10:30:10.669] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],Option[i64],str,str,f64,Option[i64])) -> boolean
[2021-03-25 10:30:10.670] [codegen] [info] generating lambda function for (Option[i64]) -> i64
[2021-03-25 10:30:10.670] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],Option[i64],str,str,f64,i64)) -> str
[2021-03-25 10:30:10.671] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],Option[i64],str,str,f64,i64,str)) -> i64
[2021-03-25 10:30:10.672] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],Option[i64],str,str,f64,i64,str)) -> boolean
[2021-03-25 10:30:10.673] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],Option[i64],str,str,f64,i64,str)) -> boolean
[2021-03-25 10:30:10.674] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],Option[i64],str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],Option[i64],f64,i64,str,str,i64)
[2021-03-25 10:30:10.702] [global] [info] Stage0 depends on: 
[2021-03-25 10:30:10.873] [global] [info] Optimization via LLVM passes took 0.161839 ms
[2021-03-25 10:30:10.944] [global] [info] Compiled code paths for stage 0 in 0.07 ms
[2021-03-25 10:30:10.944] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.242104s
[2021-03-25 10:30:10.944] [local ee] [info] split /data/zillow/Zdirty/zillow_dirty@10G.csv into 160 tasks
[2021-03-25 10:30:11.253] [E/12] [info] [Task Finished] Transform to mem in 0.309401s (11185 normal rows, 0 exceptions)
[2021-03-25 10:30:11.254] [E/11] [info] [Task Finished] Transform to mem in 0.310032s (11267 normal rows, 0 exceptions)
[2021-03-25 10:30:11.255] [driver] [info] [Task Finished] Transform to mem in 0.310473s (11167 normal rows, 0 exceptions)
[2021-03-25 10:30:11.255] [E/7] [info] [Task Finished] Transform to mem in 0.310641s (11185 normal rows, 0 exceptions)
[2021-03-25 10:30:11.255] [E/8] [info] [Task Finished] Transform to mem in 0.310796s (11177 normal rows, 0 exceptions)
[2021-03-25 10:30:11.255] [E/2] [info] [Task Finished] Transform to mem in 0.311078s (11231 normal rows, 0 exceptions)
[2021-03-25 10:30:11.255] [E/3] [info] [Task Finished] Transform to mem in 0.311142s (11335 normal rows, 0 exceptions)
[2021-03-25 10:30:11.255] [E/15] [info] [Task Finished] Transform to mem in 0.311439s (11281 normal rows, 0 exceptions)
[2021-03-25 10:30:11.256] [E/5] [info] [Task Finished] Transform to mem in 0.311787s (11189 normal rows, 0 exceptions)
[2021-03-25 10:30:11.256] [E/10] [info] [Task Finished] Transform to mem in 0.311848s (11177 normal rows, 0 exceptions)
[2021-03-25 10:30:11.257] [E/1] [info] [Task Finished] Transform to mem in 0.312742s (11287 normal rows, 0 exceptions)
[2021-03-25 10:30:11.259] [E/6] [info] [Task Finished] Transform to mem in 0.314538s (11209 normal rows, 0 exceptions)
[2021-03-25 10:30:11.260] [E/9] [info] [Task Finished] Transform to mem in 0.315988s (11335 normal rows, 0 exceptions)
[2021-03-25 10:30:11.264] [E/4] [info] [Task Finished] Transform to mem in 0.319509s (11220 normal rows, 0 exceptions)
[2021-03-25 10:30:11.264] [E/13] [info] [Task Finished] Transform to mem in 0.319865s (11220 normal rows, 0 exceptions)
[2021-03-25 10:30:11.265] [E/14] [info] [Task Finished] Transform to mem in 0.321209s (11261 normal rows, 0 exceptions)
[2021-03-25 10:30:11.562] [driver] [info] [Task Finished] Transform to mem in 0.307012s (11335 normal rows, 0 exceptions)
[2021-03-25 10:30:11.562] [E/12] [info] [Task Finished] Transform to mem in 0.308432s (11267 normal rows, 0 exceptions)
[2021-03-25 10:30:11.562] [E/7] [info] [Task Finished] Transform to mem in 0.307524s (11209 normal rows, 0 exceptions)
[2021-03-25 10:30:11.562] [E/11] [info] [Task Finished] Transform to mem in 0.308181s (11281 normal rows, 0 exceptions)
[2021-03-25 10:30:11.563] [E/3] [info] [Task Finished] Transform to mem in 0.307744s (11220 normal rows, 0 exceptions)
[2021-03-25 10:30:11.563] [E/2] [info] [Task Finished] Transform to mem in 0.307878s (11203 normal rows, 0 exceptions)
[2021-03-25 10:30:11.563] [E/15] [info] [Task Finished] Transform to mem in 0.307986s (11185 normal rows, 0 exceptions)
[2021-03-25 10:30:11.564] [E/8] [info] [Task Finished] Transform to mem in 0.309125s (11163 normal rows, 0 exceptions)
[2021-03-25 10:30:11.564] [E/5] [info] [Task Finished] Transform to mem in 0.308197s (11267 normal rows, 0 exceptions)
[2021-03-25 10:30:11.565] [E/10] [info] [Task Finished] Transform to mem in 0.309173s (11268 normal rows, 0 exceptions)
[2021-03-25 10:30:11.566] [E/1] [info] [Task Finished] Transform to mem in 0.308782s (11293 normal rows, 0 exceptions)
[2021-03-25 10:30:11.568] [E/9] [info] [Task Finished] Transform to mem in 0.308181s (11168 normal rows, 0 exceptions)
[2021-03-25 10:30:11.570] [E/6] [info] [Task Finished] Transform to mem in 0.310940s (11259 normal rows, 0 exceptions)
[2021-03-25 10:30:11.580] [E/4] [info] [Task Finished] Transform to mem in 0.316493s (11203 normal rows, 0 exceptions)
[2021-03-25 10:30:11.581] [E/14] [info] [Task Finished] Transform to mem in 0.316113s (11172 normal rows, 0 exceptions)
[2021-03-25 10:30:11.582] [E/13] [info] [Task Finished] Transform to mem in 0.317781s (11220 normal rows, 0 exceptions)
[2021-03-25 10:30:11.866] [E/11] [info] [Task Finished] Transform to mem in 0.303974s (11259 normal rows, 0 exceptions)
[2021-03-25 10:30:11.866] [E/12] [info] [Task Finished] Transform to mem in 0.304487s (11252 normal rows, 0 exceptions)
[2021-03-25 10:30:11.867] [E/7] [info] [Task Finished] Transform to mem in 0.304650s (11309 normal rows, 0 exceptions)
[2021-03-25 10:30:11.868] [E/2] [info] [Task Finished] Transform to mem in 0.304556s (11184 normal rows, 0 exceptions)
[2021-03-25 10:30:11.868] [E/3] [info] [Task Finished] Transform to mem in 0.304812s (11168 normal rows, 0 exceptions)
[2021-03-25 10:30:11.868] [driver] [info] [Task Finished] Transform to mem in 0.306615s (11280 normal rows, 0 exceptions)
[2021-03-25 10:30:11.869] [E/15] [info] [Task Finished] Transform to mem in 0.305102s (11210 normal rows, 0 exceptions)
[2021-03-25 10:30:11.870] [E/8] [info] [Task Finished] Transform to mem in 0.305930s (11201 normal rows, 0 exceptions)
[2021-03-25 10:30:11.871] [E/5] [info] [Task Finished] Transform to mem in 0.306781s (11244 normal rows, 0 exceptions)
[2021-03-25 10:30:11.871] [E/10] [info] [Task Finished] Transform to mem in 0.305791s (11288 normal rows, 0 exceptions)
[2021-03-25 10:30:11.872] [E/1] [info] [Task Finished] Transform to mem in 0.305931s (11309 normal rows, 0 exceptions)
[2021-03-25 10:30:11.873] [E/9] [info] [Task Finished] Transform to mem in 0.304897s (11259 normal rows, 0 exceptions)
[2021-03-25 10:30:11.878] [E/6] [info] [Task Finished] Transform to mem in 0.308738s (11168 normal rows, 0 exceptions)
[2021-03-25 10:30:11.894] [E/4] [info] [Task Finished] Transform to mem in 0.313736s (11184 normal rows, 0 exceptions)
[2021-03-25 10:30:11.894] [E/14] [info] [Task Finished] Transform to mem in 0.312696s (11210 normal rows, 0 exceptions)
[2021-03-25 10:30:11.895] [E/13] [info] [Task Finished] Transform to mem in 0.313453s (11192 normal rows, 0 exceptions)
[2021-03-25 10:30:12.170] [E/11] [info] [Task Finished] Transform to mem in 0.303644s (11253 normal rows, 0 exceptions)
[2021-03-25 10:30:12.171] [E/12] [info] [Task Finished] Transform to mem in 0.304429s (11285 normal rows, 0 exceptions)
[2021-03-25 10:30:12.171] [E/2] [info] [Task Finished] Transform to mem in 0.303833s (11259 normal rows, 0 exceptions)
[2021-03-25 10:30:12.171] [E/7] [info] [Task Finished] Transform to mem in 0.304520s (11312 normal rows, 0 exceptions)
[2021-03-25 10:30:12.172] [E/3] [info] [Task Finished] Transform to mem in 0.304294s (11157 normal rows, 0 exceptions)
[2021-03-25 10:30:12.173] [E/15] [info] [Task Finished] Transform to mem in 0.304328s (11210 normal rows, 0 exceptions)
[2021-03-25 10:30:12.174] [driver] [info] [Task Finished] Transform to mem in 0.305624s (11195 normal rows, 0 exceptions)
[2021-03-25 10:30:12.176] [E/8] [info] [Task Finished] Transform to mem in 0.305647s (11192 normal rows, 0 exceptions)
[2021-03-25 10:30:12.176] [E/10] [info] [Task Finished] Transform to mem in 0.305521s (11285 normal rows, 0 exceptions)
[2021-03-25 10:30:12.178] [E/1] [info] [Task Finished] Transform to mem in 0.305990s (11312 normal rows, 0 exceptions)
[2021-03-25 10:30:12.178] [E/5] [info] [Task Finished] Transform to mem in 0.306743s (11253 normal rows, 0 exceptions)
[2021-03-25 10:30:12.178] [E/9] [info] [Task Finished] Transform to mem in 0.304452s (11251 normal rows, 0 exceptions)
[2021-03-25 10:30:12.185] [E/6] [info] [Task Finished] Transform to mem in 0.306957s (11156 normal rows, 0 exceptions)
[2021-03-25 10:30:12.207] [E/4] [info] [Task Finished] Transform to mem in 0.312996s (11188 normal rows, 0 exceptions)
[2021-03-25 10:30:12.207] [E/14] [info] [Task Finished] Transform to mem in 0.312975s (11226 normal rows, 0 exceptions)
[2021-03-25 10:30:12.209] [E/13] [info] [Task Finished] Transform to mem in 0.314305s (11192 normal rows, 0 exceptions)
[2021-03-25 10:30:12.473] [E/11] [info] [Task Finished] Transform to mem in 0.303471s (11243 normal rows, 0 exceptions)
[2021-03-25 10:30:12.475] [E/12] [info] [Task Finished] Transform to mem in 0.304454s (11295 normal rows, 0 exceptions)
[2021-03-25 10:30:12.475] [E/2] [info] [Task Finished] Transform to mem in 0.303986s (11263 normal rows, 0 exceptions)
[2021-03-25 10:30:12.476] [E/7] [info] [Task Finished] Transform to mem in 0.304227s (11268 normal rows, 0 exceptions)
[2021-03-25 10:30:12.476] [E/3] [info] [Task Finished] Transform to mem in 0.304147s (11188 normal rows, 0 exceptions)
[2021-03-25 10:30:12.478] [E/15] [info] [Task Finished] Transform to mem in 0.304561s (11188 normal rows, 0 exceptions)
[2021-03-25 10:30:12.480] [driver] [info] [Task Finished] Transform to mem in 0.306266s (11226 normal rows, 0 exceptions)
[2021-03-25 10:30:12.480] [E/8] [info] [Task Finished] Transform to mem in 0.304893s (11182 normal rows, 0 exceptions)
[2021-03-25 10:30:12.481] [E/10] [info] [Task Finished] Transform to mem in 0.304670s (11253 normal rows, 0 exceptions)
[2021-03-25 10:30:12.483] [E/9] [info] [Task Finished] Transform to mem in 0.304952s (11268 normal rows, 0 exceptions)
[2021-03-25 10:30:12.484] [E/1] [info] [Task Finished] Transform to mem in 0.306249s (11278 normal rows, 0 exceptions)
[2021-03-25 10:30:12.485] [E/5] [info] [Task Finished] Transform to mem in 0.307215s (11280 normal rows, 0 exceptions)
[2021-03-25 10:30:12.493] [E/6] [info] [Task Finished] Transform to mem in 0.308126s (11188 normal rows, 0 exceptions)
[2021-03-25 10:30:12.523] [E/4] [info] [Task Finished] Transform to mem in 0.315902s (11188 normal rows, 0 exceptions)
[2021-03-25 10:30:12.523] [E/14] [info] [Task Finished] Transform to mem in 0.315667s (11208 normal rows, 0 exceptions)
[2021-03-25 10:30:12.526] [E/13] [info] [Task Finished] Transform to mem in 0.316131s (11200 normal rows, 0 exceptions)
[2021-03-25 10:30:12.780] [E/11] [info] [Task Finished] Transform to mem in 0.306974s (11216 normal rows, 0 exceptions)
[2021-03-25 10:30:12.783] [E/12] [info] [Task Finished] Transform to mem in 0.307631s (11315 normal rows, 0 exceptions)
[2021-03-25 10:30:12.783] [E/7] [info] [Task Finished] Transform to mem in 0.307363s (11268 normal rows, 0 exceptions)
[2021-03-25 10:30:12.784] [E/2] [info] [Task Finished] Transform to mem in 0.308044s (11280 normal rows, 0 exceptions)
[2021-03-25 10:30:12.784] [E/3] [info] [Task Finished] Transform to mem in 0.307470s (11168 normal rows, 0 exceptions)
[2021-03-25 10:30:12.785] [E/15] [info] [Task Finished] Transform to mem in 0.307748s (11180 normal rows, 0 exceptions)
[2021-03-25 10:30:12.789] [driver] [info] [Task Finished] Transform to mem in 0.308764s (11236 normal rows, 0 exceptions)
[2021-03-25 10:30:12.789] [E/10] [info] [Task Finished] Transform to mem in 0.307935s (11175 normal rows, 0 exceptions)
[2021-03-25 10:30:12.790] [E/8] [info] [Task Finished] Transform to mem in 0.309140s (11200 normal rows, 0 exceptions)
[2021-03-25 10:30:12.791] [E/9] [info] [Task Finished] Transform to mem in 0.308760s (11356 normal rows, 0 exceptions)
[2021-03-25 10:30:12.793] [E/1] [info] [Task Finished] Transform to mem in 0.309537s (11280 normal rows, 0 exceptions)
[2021-03-25 10:30:12.796] [E/5] [info] [Task Finished] Transform to mem in 0.310874s (11253 normal rows, 0 exceptions)
[2021-03-25 10:30:12.804] [E/6] [info] [Task Finished] Transform to mem in 0.310496s (11183 normal rows, 0 exceptions)
[2021-03-25 10:30:12.835] [E/14] [info] [Task Finished] Transform to mem in 0.312559s (11236 normal rows, 0 exceptions)
[2021-03-25 10:30:12.836] [E/4] [info] [Task Finished] Transform to mem in 0.312965s (11180 normal rows, 0 exceptions)
[2021-03-25 10:30:12.839] [E/13] [info] [Task Finished] Transform to mem in 0.313698s (11200 normal rows, 0 exceptions)
[2021-03-25 10:30:13.085] [E/11] [info] [Task Finished] Transform to mem in 0.304191s (11175 normal rows, 0 exceptions)
[2021-03-25 10:30:13.087] [E/7] [info] [Task Finished] Transform to mem in 0.304253s (11273 normal rows, 0 exceptions)
[2021-03-25 10:30:13.088] [E/12] [info] [Task Finished] Transform to mem in 0.304680s (11356 normal rows, 0 exceptions)
[2021-03-25 10:30:13.088] [E/3] [info] [Task Finished] Transform to mem in 0.304112s (11183 normal rows, 0 exceptions)
[2021-03-25 10:30:13.088] [E/2] [info] [Task Finished] Transform to mem in 0.304646s (11260 normal rows, 0 exceptions)
[2021-03-25 10:30:13.090] [E/15] [info] [Task Finished] Transform to mem in 0.305062s (11180 normal rows, 0 exceptions)
[2021-03-25 10:30:13.094] [E/10] [info] [Task Finished] Transform to mem in 0.304703s (11196 normal rows, 0 exceptions)
[2021-03-25 10:30:13.095] [E/8] [info] [Task Finished] Transform to mem in 0.304877s (11175 normal rows, 0 exceptions)
[2021-03-25 10:30:13.095] [driver] [info] [Task Finished] Transform to mem in 0.305928s (11220 normal rows, 0 exceptions)
[2021-03-25 10:30:13.097] [E/9] [info] [Task Finished] Transform to mem in 0.305148s (11376 normal rows, 0 exceptions)
[2021-03-25 10:30:13.099] [E/1] [info] [Task Finished] Transform to mem in 0.305711s (11181 normal rows, 0 exceptions)
[2021-03-25 10:30:13.103] [E/5] [info] [Task Finished] Transform to mem in 0.307448s (11352 normal rows, 0 exceptions)
[2021-03-25 10:30:13.111] [E/6] [info] [Task Finished] Transform to mem in 0.307011s (11173 normal rows, 0 exceptions)
[2021-03-25 10:30:13.149] [E/14] [info] [Task Finished] Transform to mem in 0.313504s (11190 normal rows, 0 exceptions)
[2021-03-25 10:30:13.149] [E/4] [info] [Task Finished] Transform to mem in 0.313125s (11183 normal rows, 0 exceptions)
[2021-03-25 10:30:13.154] [E/13] [info] [Task Finished] Transform to mem in 0.314147s (11233 normal rows, 0 exceptions)
[2021-03-25 10:30:13.389] [E/11] [info] [Task Finished] Transform to mem in 0.304711s (11153 normal rows, 0 exceptions)
[2021-03-25 10:30:13.393] [E/2] [info] [Task Finished] Transform to mem in 0.304345s (11181 normal rows, 0 exceptions)
[2021-03-25 10:30:13.393] [E/3] [info] [Task Finished] Transform to mem in 0.305121s (11348 normal rows, 0 exceptions)
[2021-03-25 10:30:13.393] [E/7] [info] [Task Finished] Transform to mem in 0.305690s (11349 normal rows, 0 exceptions)
[2021-03-25 10:30:13.393] [E/12] [info] [Task Finished] Transform to mem in 0.305727s (11226 normal rows, 0 exceptions)
[2021-03-25 10:30:13.395] [E/15] [info] [Task Finished] Transform to mem in 0.305112s (11185 normal rows, 0 exceptions)
[2021-03-25 10:30:13.400] [E/10] [info] [Task Finished] Transform to mem in 0.305932s (11188 normal rows, 0 exceptions)
[2021-03-25 10:30:13.401] [E/8] [info] [Task Finished] Transform to mem in 0.306285s (11233 normal rows, 0 exceptions)
[2021-03-25 10:30:13.401] [driver] [info] [Task Finished] Transform to mem in 0.306628s (11153 normal rows, 0 exceptions)
[2021-03-25 10:30:13.403] [E/9] [info] [Task Finished] Transform to mem in 0.306091s (11349 normal rows, 0 exceptions)
[2021-03-25 10:30:13.406] [E/1] [info] [Task Finished] Transform to mem in 0.306766s (11224 normal rows, 0 exceptions)
[2021-03-25 10:30:13.410] [E/5] [info] [Task Finished] Transform to mem in 0.306930s (11350 normal rows, 0 exceptions)
[2021-03-25 10:30:13.419] [E/6] [info] [Task Finished] Transform to mem in 0.307812s (11181 normal rows, 0 exceptions)
[2021-03-25 10:30:13.462] [E/14] [info] [Task Finished] Transform to mem in 0.313089s (11177 normal rows, 0 exceptions)
[2021-03-25 10:30:13.462] [E/4] [info] [Task Finished] Transform to mem in 0.313524s (11196 normal rows, 0 exceptions)
[2021-03-25 10:30:13.468] [E/13] [info] [Task Finished] Transform to mem in 0.314268s (11233 normal rows, 0 exceptions)
[2021-03-25 10:30:13.694] [E/11] [info] [Task Finished] Transform to mem in 0.304366s (11153 normal rows, 0 exceptions)
[2021-03-25 10:30:13.697] [E/2] [info] [Task Finished] Transform to mem in 0.304839s (11349 normal rows, 0 exceptions)
[2021-03-25 10:30:13.697] [E/3] [info] [Task Finished] Transform to mem in 0.304453s (11224 normal rows, 0 exceptions)
[2021-03-25 10:30:13.698] [E/12] [info] [Task Finished] Transform to mem in 0.304920s (11181 normal rows, 0 exceptions)
[2021-03-25 10:30:13.699] [E/7] [info] [Task Finished] Transform to mem in 0.305675s (11350 normal rows, 0 exceptions)
[2021-03-25 10:30:13.701] [E/15] [info] [Task Finished] Transform to mem in 0.305355s (11177 normal rows, 0 exceptions)
[2021-03-25 10:30:13.705] [E/10] [info] [Task Finished] Transform to mem in 0.304878s (11190 normal rows, 0 exceptions)
[2021-03-25 10:30:13.706] [E/8] [info] [Task Finished] Transform to mem in 0.305388s (11204 normal rows, 0 exceptions)
[2021-03-25 10:30:13.708] [E/9] [info] [Task Finished] Transform to mem in 0.305312s (11354 normal rows, 0 exceptions)
[2021-03-25 10:30:13.709] [driver] [info] [Task Finished] Transform to mem in 0.307712s (11183 normal rows, 0 exceptions)
[2021-03-25 10:30:13.713] [E/1] [info] [Task Finished] Transform to mem in 0.306680s (11224 normal rows, 0 exceptions)
[2021-03-25 10:30:13.717] [E/5] [info] [Task Finished] Transform to mem in 0.307315s (11338 normal rows, 0 exceptions)
[2021-03-25 10:30:13.727] [E/6] [info] [Task Finished] Transform to mem in 0.307904s (11193 normal rows, 0 exceptions)
[2021-03-25 10:30:13.775] [E/14] [info] [Task Finished] Transform to mem in 0.312878s (11177 normal rows, 0 exceptions)
[2021-03-25 10:30:13.775] [E/4] [info] [Task Finished] Transform to mem in 0.312837s (11190 normal rows, 0 exceptions)
[2021-03-25 10:30:13.782] [E/13] [info] [Task Finished] Transform to mem in 0.313799s (11204 normal rows, 0 exceptions)
[2021-03-25 10:30:13.998] [E/11] [info] [Task Finished] Transform to mem in 0.304247s (11183 normal rows, 0 exceptions)
[2021-03-25 10:30:14.003] [E/3] [info] [Task Finished] Transform to mem in 0.305317s (11208 normal rows, 0 exceptions)
[2021-03-25 10:30:14.003] [E/12] [info] [Task Finished] Transform to mem in 0.304910s (11354 normal rows, 0 exceptions)
[2021-03-25 10:30:14.004] [E/7] [info] [Task Finished] Transform to mem in 0.304971s (11181 normal rows, 0 exceptions)
[2021-03-25 10:30:14.006] [E/2] [info] [Task Finished] Transform to mem in 0.308893s (11354 normal rows, 0 exceptions)
[2021-03-25 10:30:14.007] [E/15] [info] [Task Finished] Transform to mem in 0.305763s (11189 normal rows, 0 exceptions)
[2021-03-25 10:30:14.010] [E/10] [info] [Task Finished] Transform to mem in 0.305400s (11180 normal rows, 0 exceptions)
[2021-03-25 10:30:14.013] [E/8] [info] [Task Finished] Transform to mem in 0.306551s (11214 normal rows, 0 exceptions)
[2021-03-25 10:30:14.013] [E/9] [info] [Task Finished] Transform to mem in 0.305097s (11180 normal rows, 0 exceptions)
[2021-03-25 10:30:14.016] [driver] [info] [Task Finished] Transform to mem in 0.306478s (11273 normal rows, 0 exceptions)
[2021-03-25 10:30:14.020] [E/1] [info] [Task Finished] Transform to mem in 0.307359s (11287 normal rows, 0 exceptions)
[2021-03-25 10:30:14.025] [E/5] [info] [Task Finished] Transform to mem in 0.307946s (11359 normal rows, 0 exceptions)
[2021-03-25 10:30:14.035] [E/6] [info] [Task Finished] Transform to mem in 0.308269s (11181 normal rows, 0 exceptions)
[2021-03-25 10:30:14.089] [E/14] [info] [Task Finished] Transform to mem in 0.314191s (11167 normal rows, 0 exceptions)
[2021-03-25 10:30:14.090] [E/4] [info] [Task Finished] Transform to mem in 0.314650s (11201 normal rows, 0 exceptions)
[2021-03-25 10:30:14.134] [E/13] [info] [Task Finished] Transform to mem in 0.352221s (12533 normal rows, 0 exceptions)
[2021-03-25 10:30:14.134] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 3.19028s
[2021-03-25 10:30:14.134] [global] [info] [Transform Stage] Stage 0 total wall clock time: 49.3585s, 56350770 input rows, time to process 1 row via fast path: 0.000875915ms

[2021-03-25 10:30:14.173] [global] [info] Writing 1798491 rows as output to file (273.80 MB)
[2021-03-25 10:30:14.201] [E/13] [info] [Task Finished] write to file in 0.028480s (102395 rows, 15.59 MB)
[2021-03-25 10:30:14.204] [E/8] [info] [Task Finished] write to file in 0.031254s (112425 rows, 17.11 MB)
[2021-03-25 10:30:14.204] [E/10] [info] [Task Finished] write to file in 0.031548s (112421 rows, 17.11 MB)
[2021-03-25 10:30:14.206] [E/7] [info] [Task Finished] write to file in 0.033102s (123508 rows, 18.80 MB)
[2021-03-25 10:30:14.206] [E/3] [info] [Task Finished] write to file in 0.033091s (123458 rows, 18.80 MB)
[2021-03-25 10:30:14.206] [E/12] [info] [Task Finished] write to file in 0.033133s (123474 rows, 18.80 MB)
[2021-03-25 10:30:14.206] [E/2] [info] [Task Finished] write to file in 0.033263s (123570 rows, 18.81 MB)
[2021-03-25 10:30:14.206] [E/5] [info] [Task Finished] write to file in 0.033299s (123479 rows, 18.80 MB)
[2021-03-25 10:30:14.206] [E/15] [info] [Task Finished] write to file in 0.033325s (123413 rows, 18.79 MB)
[2021-03-25 10:30:14.206] [E/9] [info] [Task Finished] write to file in 0.033362s (123409 rows, 18.79 MB)
[2021-03-25 10:30:14.206] [E/11] [info] [Task Finished] write to file in 0.033486s (123474 rows, 18.80 MB)
[2021-03-25 10:30:14.206] [driver] [info] [Task Finished] write to file in 0.033500s (123598 rows, 18.82 MB)
[2021-03-25 10:30:14.206] [E/4] [info] [Task Finished] write to file in 0.033520s (123752 rows, 18.84 MB)
[2021-03-25 10:30:14.206] [E/6] [info] [Task Finished] write to file in 0.033607s (123704 rows, 18.83 MB)
[2021-03-25 10:30:14.210] [E/1] [info] [Task Finished] write to file in 0.036740s (112411 rows, 17.11 MB)
[2021-03-25 10:30:14.210] [global] [info] writing output took 0.075595s
[2021-03-25 10:30:14.210] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.0756758s
[2021-03-25 10:30:14.210] [global] [info] [Transform Stage] Stage 0 took 3.50811s
[2021-03-25 10:30:14.210] [global] [info] Query Execution took 3.5869s. (planning: 0.078754s, execution: 3.50815s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 23 2021, 11:08:57) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Zdirty/zillow_dirty@10G.csv']
>>> bug in generated parser, force to false
Tuplex startup time: 0.005995273590087891
>>> running pipeline with custom resolve logic...
Tuplex job time: 3.9039552211761475 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 15, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '10G', 'tuplex.env.hostname': '4e1bb76d17c5', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'False', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '64MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.0337648,"llvm_optimization_time_s":0.161839,"llvm_compilation_time_s":0.0707862,"total_compilation_time_s":0.242101,"sampling_time_s":0.110699,"stages":[{"stage_no":0,"fast_path_wall_time_s":49.3585,"fast_path_time_s":3.1903,"fast_path_per_row_time_ns":875.915,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"webui.enable": false, "executorMemory": "6G", "executorCount": 15, "driverMemory": "10G", "partitionSize": "32MB", "runTimeMemory": "64MB", "inputSplitSize": "64MB", "useLLVMOptimizer": true, "optimizer.nullValueOptimization": false, "csv.selectionPushdown": true, "optimizer.generateParser": false, "optimizer.mergeExceptionsInOrder": false, "optimizer.filterPushdown": true, "resolveWithInterpreterOnly": false}
{"startupTime": 0.005995273590087891, "jobTime": 3.9039552211761475, "mode": "custom", "mergeExceptionsInOrder": false}
[2021-03-25 10:30:14.234] [local execution engine] [info] Stopped local executor E/1
[2021-03-25 10:30:14.235] [local execution engine] [info] Stopped local executor E/2
[2021-03-25 10:30:14.237] [local execution engine] [info] Stopped local executor E/3
[2021-03-25 10:30:14.238] [local execution engine] [info] Stopped local executor E/4
[2021-03-25 10:30:14.239] [local execution engine] [info] Stopped local executor E/5
[2021-03-25 10:30:14.240] [local execution engine] [info] Stopped local executor E/6
[2021-03-25 10:30:14.242] [local execution engine] [info] Stopped local executor E/7
[2021-03-25 10:30:14.243] [local execution engine] [info] Stopped local executor E/8
[2021-03-25 10:30:14.244] [local execution engine] [info] Stopped local executor E/9
[2021-03-25 10:30:14.245] [local execution engine] [info] Stopped local executor E/10
[2021-03-25 10:30:14.246] [local execution engine] [info] Stopped local executor E/11
[2021-03-25 10:30:14.248] [local execution engine] [info] Stopped local executor E/12
[2021-03-25 10:30:14.249] [local execution engine] [info] Stopped local executor E/13
[2021-03-25 10:30:14.250] [local execution engine] [info] Stopped local executor E/14
[2021-03-25 10:30:14.251] [local execution engine] [info] Stopped local executor E/15
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
