[2021-03-31 18:13:46.924] [global] [error] could not find backend, defaulting to local
[2021-03-31 18:13:46.924] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2021-03-31 18:13:46.924] [local ee] [info] initializing LLVM backend
[2021-03-31 18:13:46.924] [local ee] [warning] init JIT compiler also only in local mode
[2021-03-31 18:13:46.925] [LLVM] [info] compiling code for skylake-avx512
[2021-03-31 18:13:46.925] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.925] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.925] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.925] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.925] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.925] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.925] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.925] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.925] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.925] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.926] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.926] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.926] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.926] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.926] [memory] [info] allocated bitmap managed memory region (6.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/1 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/1] [info] starting detached process queue
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/2 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/2] [info] starting detached process queue
[2021-03-31 18:13:46.926] [E/1] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [E/2] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/3 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/3] [info] starting detached process queue
[2021-03-31 18:13:46.926] [E/3] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/4 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/4] [info] starting detached process queue
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/5 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/5] [info] starting detached process queue
[2021-03-31 18:13:46.926] [E/4] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/6 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/5] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [E/6] [info] starting detached process queue
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/7 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/7] [info] starting detached process queue
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/8 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/6] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [E/7] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [E/8] [info] starting detached process queue
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/9 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/8] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [E/9] [info] starting detached process queue
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/10 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/9] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [E/10] [info] starting detached process queue
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/11 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/11] [info] starting detached process queue
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/12 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/10] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [E/11] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [E/12] [info] starting detached process queue
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/13 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/13] [info] starting detached process queue
[2021-03-31 18:13:46.926] [E/12] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/14 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [E/13] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [E/14] [info] starting detached process queue
[2021-03-31 18:13:46.926] [local execution engine] [info] started local executor E/15 (6.00 GB, 32.00 MB default partition size)
[2021-03-31 18:13:46.926] [memory] [info] allocated bitmap managed memory region (10.00 GB, 32.00 MB block size)
[2021-03-31 18:13:46.926] [E/15] [info] starting detached process queue
[2021-03-31 18:13:46.926] [local execution engine] [info] started driver (10.00 GB, 32.00 MB default partition size)
overflow will be cached at file:///tmp/driver
[2021-03-31 18:13:46.926] [E/14] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [E/15] [info] initialized runtime memory (4.00 MB)
[2021-03-31 18:13:46.926] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2021-03-31 18:13:46.927] [global] [info] sampled file:///data/zillow/Z2_preprocessed/zillow_Z2_10G.csv on 256.00 KB
[2021-03-31 18:13:47.062] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:47.078] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:47.086] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:47.113] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:47.133] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:47.146] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:47.179] [global] [info] performing static typing for UDF in operator withColumn
[2021-03-31 18:13:47.196] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:13:47.198] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:13:47.201] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:13:47.203] [global] [info] performing static typing for UDF in operator map
[2021-03-31 18:13:47.219] [logical planner] [info] logical optimization took 0.020372ms
output type of specialized pipeline is: (str)
is this the most outer stage?: 1
[2021-03-31 18:13:47.219] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (13 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:13:47.220] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2021-03-31 18:13:47.221] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2021-03-31 18:13:47.221] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2021-03-31 18:13:47.222] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2021-03-31 18:13:47.222] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2021-03-31 18:13:47.223] [codegen] [info] generating lambda function for (Option[str]) -> str
[2021-03-31 18:13:47.224] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> f64
[2021-03-31 18:13:47.225] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64)) -> i64
[2021-03-31 18:13:47.226] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64)) -> str
[2021-03-31 18:13:47.226] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64,str)) -> i64
[2021-03-31 18:13:47.227] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:13:47.228] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2021-03-31 18:13:47.229] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2021-03-31 18:13:47.254] [global] [info] Stage0 depends on: 
[2021-03-31 18:13:47.399] [global] [info] Optimization via LLVM passes took 0.137375 ms
[2021-03-31 18:13:47.461] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2021-03-31 18:13:47.461] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.206824s
[2021-03-31 18:13:47.461] [local ee] [info] split /data/zillow/Z2_preprocessed/zillow_Z2_10G.csv into 160 tasks
[2021-03-31 18:13:47.717] [E/13] [info] [Task Finished] Transform to mem in 0.255317s (13953 normal rows, 0 exceptions)
[2021-03-31 18:13:47.717] [E/14] [info] [Task Finished] Transform to mem in 0.255827s (14020 normal rows, 0 exceptions)
[2021-03-31 18:13:47.717] [E/4] [info] [Task Finished] Transform to mem in 0.255832s (13920 normal rows, 0 exceptions)
[2021-03-31 18:13:47.717] [E/11] [info] [Task Finished] Transform to mem in 0.255851s (13988 normal rows, 0 exceptions)
[2021-03-31 18:13:47.717] [E/2] [info] [Task Finished] Transform to mem in 0.255915s (14206 normal rows, 0 exceptions)
[2021-03-31 18:13:47.717] [E/10] [info] [Task Finished] Transform to mem in 0.256175s (14253 normal rows, 0 exceptions)
[2021-03-31 18:13:47.717] [E/8] [info] [Task Finished] Transform to mem in 0.256265s (14237 normal rows, 0 exceptions)
[2021-03-31 18:13:47.718] [E/9] [info] [Task Finished] Transform to mem in 0.256399s (14162 normal rows, 0 exceptions)
[2021-03-31 18:13:47.718] [E/7] [info] [Task Finished] Transform to mem in 0.256418s (14207 normal rows, 0 exceptions)
[2021-03-31 18:13:47.718] [E/1] [info] [Task Finished] Transform to mem in 0.256403s (14226 normal rows, 0 exceptions)
[2021-03-31 18:13:47.718] [E/6] [info] [Task Finished] Transform to mem in 0.256564s (14177 normal rows, 0 exceptions)
[2021-03-31 18:13:47.718] [E/15] [info] [Task Finished] Transform to mem in 0.256767s (14267 normal rows, 0 exceptions)
[2021-03-31 18:13:47.721] [driver] [info] [Task Finished] Transform to mem in 0.259790s (14217 normal rows, 0 exceptions)
[2021-03-31 18:13:47.722] [E/3] [info] [Task Finished] Transform to mem in 0.260871s (13970 normal rows, 0 exceptions)
[2021-03-31 18:13:47.723] [E/12] [info] [Task Finished] Transform to mem in 0.261522s (14248 normal rows, 0 exceptions)
[2021-03-31 18:13:47.727] [E/5] [info] [Task Finished] Transform to mem in 0.265759s (14242 normal rows, 0 exceptions)
[2021-03-31 18:13:47.969] [E/13] [info] [Task Finished] Transform to mem in 0.252902s (14241 normal rows, 0 exceptions)
[2021-03-31 18:13:47.970] [E/14] [info] [Task Finished] Transform to mem in 0.253200s (14059 normal rows, 0 exceptions)
[2021-03-31 18:13:47.971] [E/9] [info] [Task Finished] Transform to mem in 0.252921s (14086 normal rows, 0 exceptions)
[2021-03-31 18:13:47.971] [E/11] [info] [Task Finished] Transform to mem in 0.253532s (14238 normal rows, 0 exceptions)
[2021-03-31 18:13:47.971] [E/8] [info] [Task Finished] Transform to mem in 0.253155s (14221 normal rows, 0 exceptions)
[2021-03-31 18:13:47.971] [E/10] [info] [Task Finished] Transform to mem in 0.253477s (14095 normal rows, 0 exceptions)
[2021-03-31 18:13:47.971] [E/7] [info] [Task Finished] Transform to mem in 0.253240s (14097 normal rows, 0 exceptions)
[2021-03-31 18:13:47.971] [E/1] [info] [Task Finished] Transform to mem in 0.253357s (14212 normal rows, 0 exceptions)
[2021-03-31 18:13:47.971] [E/6] [info] [Task Finished] Transform to mem in 0.253347s (14108 normal rows, 0 exceptions)
[2021-03-31 18:13:47.971] [E/15] [info] [Task Finished] Transform to mem in 0.253340s (14100 normal rows, 0 exceptions)
[2021-03-31 18:13:47.973] [driver] [info] [Task Finished] Transform to mem in 0.252512s (14212 normal rows, 0 exceptions)
[2021-03-31 18:13:47.975] [E/3] [info] [Task Finished] Transform to mem in 0.252798s (14099 normal rows, 0 exceptions)
[2021-03-31 18:13:47.975] [E/2] [info] [Task Finished] Transform to mem in 0.258038s (14081 normal rows, 0 exceptions)
[2021-03-31 18:13:47.977] [E/4] [info] [Task Finished] Transform to mem in 0.259671s (14114 normal rows, 0 exceptions)
[2021-03-31 18:13:47.981] [E/5] [info] [Task Finished] Transform to mem in 0.254107s (14223 normal rows, 0 exceptions)
[2021-03-31 18:13:47.981] [E/12] [info] [Task Finished] Transform to mem in 0.258504s (14094 normal rows, 0 exceptions)
[2021-03-31 18:13:48.226] [E/13] [info] [Task Finished] Transform to mem in 0.256840s (14089 normal rows, 0 exceptions)
[2021-03-31 18:13:48.227] [E/14] [info] [Task Finished] Transform to mem in 0.256512s (14083 normal rows, 0 exceptions)
[2021-03-31 18:13:48.227] [E/11] [info] [Task Finished] Transform to mem in 0.256559s (14170 normal rows, 0 exceptions)
[2021-03-31 18:13:48.227] [E/8] [info] [Task Finished] Transform to mem in 0.256535s (14002 normal rows, 0 exceptions)
[2021-03-31 18:13:48.227] [E/1] [info] [Task Finished] Transform to mem in 0.256299s (13961 normal rows, 0 exceptions)
[2021-03-31 18:13:48.228] [E/9] [info] [Task Finished] Transform to mem in 0.257167s (14235 normal rows, 0 exceptions)
[2021-03-31 18:13:48.228] [E/7] [info] [Task Finished] Transform to mem in 0.256882s (14206 normal rows, 0 exceptions)
[2021-03-31 18:13:48.228] [E/10] [info] [Task Finished] Transform to mem in 0.257151s (14226 normal rows, 0 exceptions)
[2021-03-31 18:13:48.228] [E/15] [info] [Task Finished] Transform to mem in 0.256767s (14233 normal rows, 0 exceptions)
[2021-03-31 18:13:48.228] [E/6] [info] [Task Finished] Transform to mem in 0.257124s (14246 normal rows, 0 exceptions)
[2021-03-31 18:13:48.229] [driver] [info] [Task Finished] Transform to mem in 0.255404s (13915 normal rows, 0 exceptions)
[2021-03-31 18:13:48.232] [E/3] [info] [Task Finished] Transform to mem in 0.256806s (14253 normal rows, 0 exceptions)
[2021-03-31 18:13:48.237] [E/2] [info] [Task Finished] Transform to mem in 0.261790s (14267 normal rows, 0 exceptions)
[2021-03-31 18:13:48.239] [E/5] [info] [Task Finished] Transform to mem in 0.257535s (14245 normal rows, 0 exceptions)
[2021-03-31 18:13:48.239] [E/4] [info] [Task Finished] Transform to mem in 0.262623s (13892 normal rows, 0 exceptions)
[2021-03-31 18:13:48.243] [E/12] [info] [Task Finished] Transform to mem in 0.261975s (14260 normal rows, 0 exceptions)
[2021-03-31 18:13:48.483] [E/13] [info] [Task Finished] Transform to mem in 0.256493s (13978 normal rows, 0 exceptions)
[2021-03-31 18:13:48.484] [E/8] [info] [Task Finished] Transform to mem in 0.256702s (14015 normal rows, 0 exceptions)
[2021-03-31 18:13:48.484] [E/14] [info] [Task Finished] Transform to mem in 0.257323s (14181 normal rows, 0 exceptions)
[2021-03-31 18:13:48.485] [E/6] [info] [Task Finished] Transform to mem in 0.256435s (14054 normal rows, 0 exceptions)
[2021-03-31 18:13:48.485] [E/7] [info] [Task Finished] Transform to mem in 0.256953s (13982 normal rows, 0 exceptions)
[2021-03-31 18:13:48.485] [E/1] [info] [Task Finished] Transform to mem in 0.257526s (14142 normal rows, 0 exceptions)
[2021-03-31 18:13:48.485] [E/9] [info] [Task Finished] Transform to mem in 0.257260s (14280 normal rows, 0 exceptions)
[2021-03-31 18:13:48.485] [E/10] [info] [Task Finished] Transform to mem in 0.257164s (14173 normal rows, 0 exceptions)
[2021-03-31 18:13:48.485] [E/15] [info] [Task Finished] Transform to mem in 0.257282s (14235 normal rows, 0 exceptions)
[2021-03-31 18:13:48.486] [driver] [info] [Task Finished] Transform to mem in 0.256959s (14129 normal rows, 0 exceptions)
[2021-03-31 18:13:48.486] [E/11] [info] [Task Finished] Transform to mem in 0.258790s (14234 normal rows, 0 exceptions)
[2021-03-31 18:13:48.488] [E/3] [info] [Task Finished] Transform to mem in 0.256806s (14223 normal rows, 0 exceptions)
[2021-03-31 18:13:48.496] [E/5] [info] [Task Finished] Transform to mem in 0.257414s (14108 normal rows, 0 exceptions)
[2021-03-31 18:13:48.498] [E/2] [info] [Task Finished] Transform to mem in 0.261342s (14081 normal rows, 0 exceptions)
[2021-03-31 18:13:48.501] [E/4] [info] [Task Finished] Transform to mem in 0.261257s (14208 normal rows, 0 exceptions)
[2021-03-31 18:13:48.506] [E/12] [info] [Task Finished] Transform to mem in 0.262410s (14089 normal rows, 0 exceptions)
[2021-03-31 18:13:48.736] [E/13] [info] [Task Finished] Transform to mem in 0.252896s (14094 normal rows, 0 exceptions)
[2021-03-31 18:13:48.737] [E/14] [info] [Task Finished] Transform to mem in 0.253009s (14088 normal rows, 0 exceptions)
[2021-03-31 18:13:48.737] [E/8] [info] [Task Finished] Transform to mem in 0.253346s (14232 normal rows, 0 exceptions)
[2021-03-31 18:13:48.738] [E/6] [info] [Task Finished] Transform to mem in 0.252824s (14106 normal rows, 0 exceptions)
[2021-03-31 18:13:48.738] [E/1] [info] [Task Finished] Transform to mem in 0.253023s (14113 normal rows, 0 exceptions)
[2021-03-31 18:13:48.738] [E/9] [info] [Task Finished] Transform to mem in 0.253330s (14075 normal rows, 0 exceptions)
[2021-03-31 18:13:48.738] [E/15] [info] [Task Finished] Transform to mem in 0.253094s (14095 normal rows, 0 exceptions)
[2021-03-31 18:13:48.739] [E/7] [info] [Task Finished] Transform to mem in 0.253913s (14211 normal rows, 0 exceptions)
[2021-03-31 18:13:48.739] [E/10] [info] [Task Finished] Transform to mem in 0.253564s (14223 normal rows, 0 exceptions)
[2021-03-31 18:13:48.739] [driver] [info] [Task Finished] Transform to mem in 0.253064s (14086 normal rows, 0 exceptions)
[2021-03-31 18:13:48.740] [E/11] [info] [Task Finished] Transform to mem in 0.253775s (14226 normal rows, 0 exceptions)
[2021-03-31 18:13:48.742] [E/3] [info] [Task Finished] Transform to mem in 0.253115s (14170 normal rows, 0 exceptions)
[2021-03-31 18:13:48.750] [E/5] [info] [Task Finished] Transform to mem in 0.253553s (14002 normal rows, 0 exceptions)
[2021-03-31 18:13:48.757] [E/2] [info] [Task Finished] Transform to mem in 0.258573s (14226 normal rows, 0 exceptions)
[2021-03-31 18:13:48.759] [E/4] [info] [Task Finished] Transform to mem in 0.257942s (14206 normal rows, 0 exceptions)
[2021-03-31 18:13:48.764] [E/12] [info] [Task Finished] Transform to mem in 0.258237s (13970 normal rows, 0 exceptions)
[2021-03-31 18:13:48.990] [E/8] [info] [Task Finished] Transform to mem in 0.253071s (13920 normal rows, 0 exceptions)
[2021-03-31 18:13:48.991] [E/6] [info] [Task Finished] Transform to mem in 0.253706s (14267 normal rows, 0 exceptions)
[2021-03-31 18:13:48.992] [E/1] [info] [Task Finished] Transform to mem in 0.253632s (14248 normal rows, 0 exceptions)
[2021-03-31 18:13:48.992] [E/9] [info] [Task Finished] Transform to mem in 0.253246s (13912 normal rows, 0 exceptions)
[2021-03-31 18:13:48.992] [E/14] [info] [Task Finished] Transform to mem in 0.254552s (14233 normal rows, 0 exceptions)
[2021-03-31 18:13:48.992] [E/10] [info] [Task Finished] Transform to mem in 0.253239s (13988 normal rows, 0 exceptions)
[2021-03-31 18:13:48.992] [driver] [info] [Task Finished] Transform to mem in 0.253178s (14171 normal rows, 0 exceptions)
[2021-03-31 18:13:48.992] [E/15] [info] [Task Finished] Transform to mem in 0.253821s (14248 normal rows, 0 exceptions)
[2021-03-31 18:13:48.993] [E/13] [info] [Task Finished] Transform to mem in 0.256803s (14237 normal rows, 0 exceptions)
[2021-03-31 18:13:48.993] [E/7] [info] [Task Finished] Transform to mem in 0.254022s (14237 normal rows, 0 exceptions)
[2021-03-31 18:13:48.993] [E/11] [info] [Task Finished] Transform to mem in 0.253650s (14234 normal rows, 0 exceptions)
[2021-03-31 18:13:48.994] [E/3] [info] [Task Finished] Transform to mem in 0.252646s (14015 normal rows, 0 exceptions)
[2021-03-31 18:13:49.004] [E/5] [info] [Task Finished] Transform to mem in 0.254148s (14176 normal rows, 0 exceptions)
[2021-03-31 18:13:49.015] [E/2] [info] [Task Finished] Transform to mem in 0.258335s (14246 normal rows, 0 exceptions)
[2021-03-31 18:13:49.016] [E/4] [info] [Task Finished] Transform to mem in 0.257824s (14059 normal rows, 0 exceptions)
[2021-03-31 18:13:49.023] [E/12] [info] [Task Finished] Transform to mem in 0.258860s (14114 normal rows, 0 exceptions)
[2021-03-31 18:13:49.244] [E/8] [info] [Task Finished] Transform to mem in 0.253669s (14238 normal rows, 0 exceptions)
[2021-03-31 18:13:49.244] [E/6] [info] [Task Finished] Transform to mem in 0.253218s (14081 normal rows, 0 exceptions)
[2021-03-31 18:13:49.245] [E/14] [info] [Task Finished] Transform to mem in 0.253514s (14086 normal rows, 0 exceptions)
[2021-03-31 18:13:49.245] [driver] [info] [Task Finished] Transform to mem in 0.253055s (14218 normal rows, 0 exceptions)
[2021-03-31 18:13:49.245] [E/1] [info] [Task Finished] Transform to mem in 0.253753s (14095 normal rows, 0 exceptions)
[2021-03-31 18:13:49.245] [E/10] [info] [Task Finished] Transform to mem in 0.253251s (14091 normal rows, 0 exceptions)
[2021-03-31 18:13:49.245] [E/9] [info] [Task Finished] Transform to mem in 0.253826s (14221 normal rows, 0 exceptions)
[2021-03-31 18:13:49.246] [E/15] [info] [Task Finished] Transform to mem in 0.253711s (14095 normal rows, 0 exceptions)
[2021-03-31 18:13:49.246] [E/13] [info] [Task Finished] Transform to mem in 0.253708s (14113 normal rows, 0 exceptions)
[2021-03-31 18:13:49.247] [E/7] [info] [Task Finished] Transform to mem in 0.254288s (14197 normal rows, 0 exceptions)
[2021-03-31 18:13:49.247] [E/11] [info] [Task Finished] Transform to mem in 0.253560s (14114 normal rows, 0 exceptions)
[2021-03-31 18:13:49.248] [E/3] [info] [Task Finished] Transform to mem in 0.253539s (14094 normal rows, 0 exceptions)
[2021-03-31 18:13:49.258] [E/5] [info] [Task Finished] Transform to mem in 0.254343s (14197 normal rows, 0 exceptions)
[2021-03-31 18:13:49.273] [E/2] [info] [Task Finished] Transform to mem in 0.258189s (14115 normal rows, 0 exceptions)
[2021-03-31 18:13:49.275] [E/4] [info] [Task Finished] Transform to mem in 0.258136s (14083 normal rows, 0 exceptions)
[2021-03-31 18:13:49.282] [E/12] [info] [Task Finished] Transform to mem in 0.259332s (14235 normal rows, 0 exceptions)
[2021-03-31 18:13:49.497] [E/8] [info] [Task Finished] Transform to mem in 0.253244s (14144 normal rows, 0 exceptions)
[2021-03-31 18:13:49.497] [E/6] [info] [Task Finished] Transform to mem in 0.252909s (14015 normal rows, 0 exceptions)
[2021-03-31 18:13:49.498] [driver] [info] [Task Finished] Transform to mem in 0.252694s (14159 normal rows, 0 exceptions)
[2021-03-31 18:13:49.498] [E/1] [info] [Task Finished] Transform to mem in 0.253091s (14010 normal rows, 0 exceptions)
[2021-03-31 18:13:49.499] [E/10] [info] [Task Finished] Transform to mem in 0.253477s (14246 normal rows, 0 exceptions)
[2021-03-31 18:13:49.499] [E/9] [info] [Task Finished] Transform to mem in 0.253495s (14178 normal rows, 0 exceptions)
[2021-03-31 18:13:49.499] [E/14] [info] [Task Finished] Transform to mem in 0.253872s (14237 normal rows, 0 exceptions)
[2021-03-31 18:13:49.499] [E/15] [info] [Task Finished] Transform to mem in 0.253172s (13970 normal rows, 0 exceptions)
[2021-03-31 18:13:49.500] [E/11] [info] [Task Finished] Transform to mem in 0.252942s (13901 normal rows, 0 exceptions)
[2021-03-31 18:13:49.501] [E/13] [info] [Task Finished] Transform to mem in 0.254259s (14253 normal rows, 0 exceptions)
[2021-03-31 18:13:49.501] [E/7] [info] [Task Finished] Transform to mem in 0.253561s (14236 normal rows, 0 exceptions)
[2021-03-31 18:13:49.502] [E/3] [info] [Task Finished] Transform to mem in 0.254005s (14267 normal rows, 0 exceptions)
[2021-03-31 18:13:49.512] [E/5] [info] [Task Finished] Transform to mem in 0.253727s (14248 normal rows, 0 exceptions)
[2021-03-31 18:13:49.531] [E/2] [info] [Task Finished] Transform to mem in 0.257704s (13953 normal rows, 0 exceptions)
[2021-03-31 18:13:49.533] [E/4] [info] [Task Finished] Transform to mem in 0.258789s (14207 normal rows, 0 exceptions)
[2021-03-31 18:13:49.541] [E/12] [info] [Task Finished] Transform to mem in 0.258954s (14245 normal rows, 0 exceptions)
[2021-03-31 18:13:49.751] [E/8] [info] [Task Finished] Transform to mem in 0.253241s (13999 normal rows, 0 exceptions)
[2021-03-31 18:13:49.751] [driver] [info] [Task Finished] Transform to mem in 0.253141s (14242 normal rows, 0 exceptions)
[2021-03-31 18:13:49.751] [E/6] [info] [Task Finished] Transform to mem in 0.253707s (14158 normal rows, 0 exceptions)
[2021-03-31 18:13:49.752] [E/1] [info] [Task Finished] Transform to mem in 0.253343s (14020 normal rows, 0 exceptions)
[2021-03-31 18:13:49.753] [E/14] [info] [Task Finished] Transform to mem in 0.253431s (14054 normal rows, 0 exceptions)
[2021-03-31 18:13:49.753] [E/10] [info] [Task Finished] Transform to mem in 0.253984s (14173 normal rows, 0 exceptions)
[2021-03-31 18:13:49.753] [E/9] [info] [Task Finished] Transform to mem in 0.253938s (14235 normal rows, 0 exceptions)
[2021-03-31 18:13:49.753] [E/15] [info] [Task Finished] Transform to mem in 0.253876s (14114 normal rows, 0 exceptions)
[2021-03-31 18:13:49.754] [E/13] [info] [Task Finished] Transform to mem in 0.253436s (14081 normal rows, 0 exceptions)
[2021-03-31 18:13:49.754] [E/11] [info] [Task Finished] Transform to mem in 0.254202s (14238 normal rows, 0 exceptions)
[2021-03-31 18:13:49.755] [E/7] [info] [Task Finished] Transform to mem in 0.254003s (14095 normal rows, 0 exceptions)
[2021-03-31 18:13:49.755] [E/3] [info] [Task Finished] Transform to mem in 0.253483s (14221 normal rows, 0 exceptions)
[2021-03-31 18:13:49.766] [E/5] [info] [Task Finished] Transform to mem in 0.253781s (14089 normal rows, 0 exceptions)
[2021-03-31 18:13:49.789] [E/2] [info] [Task Finished] Transform to mem in 0.258119s (14094 normal rows, 0 exceptions)
[2021-03-31 18:13:49.792] [E/4] [info] [Task Finished] Transform to mem in 0.258229s (14212 normal rows, 0 exceptions)
[2021-03-31 18:13:49.799] [E/12] [info] [Task Finished] Transform to mem in 0.258114s (14108 normal rows, 0 exceptions)
[2021-03-31 18:13:50.004] [E/8] [info] [Task Finished] Transform to mem in 0.253671s (14100 normal rows, 0 exceptions)
[2021-03-31 18:13:50.005] [driver] [info] [Task Finished] Transform to mem in 0.253383s (14217 normal rows, 0 exceptions)
[2021-03-31 18:13:50.005] [E/6] [info] [Task Finished] Transform to mem in 0.253417s (14094 normal rows, 0 exceptions)
[2021-03-31 18:13:50.005] [E/1] [info] [Task Finished] Transform to mem in 0.253156s (14094 normal rows, 0 exceptions)
[2021-03-31 18:13:50.006] [E/14] [info] [Task Finished] Transform to mem in 0.253734s (14223 normal rows, 0 exceptions)
[2021-03-31 18:13:50.007] [E/10] [info] [Task Finished] Transform to mem in 0.253753s (14089 normal rows, 0 exceptions)
[2021-03-31 18:13:50.007] [E/9] [info] [Task Finished] Transform to mem in 0.253882s (14086 normal rows, 0 exceptions)
[2021-03-31 18:13:50.008] [E/15] [info] [Task Finished] Transform to mem in 0.254463s (14232 normal rows, 0 exceptions)
[2021-03-31 18:13:50.008] [E/13] [info] [Task Finished] Transform to mem in 0.253768s (14170 normal rows, 0 exceptions)
[2021-03-31 18:13:50.009] [E/3] [info] [Task Finished] Transform to mem in 0.253548s (14206 normal rows, 0 exceptions)
[2021-03-31 18:13:50.009] [E/7] [info] [Task Finished] Transform to mem in 0.254739s (14226 normal rows, 0 exceptions)
[2021-03-31 18:13:50.016] [E/11] [info] [Task Finished] Transform to mem in 0.262108s (14002 normal rows, 0 exceptions)
[2021-03-31 18:13:50.020] [E/5] [info] [Task Finished] Transform to mem in 0.253933s (13961 normal rows, 0 exceptions)
[2021-03-31 18:13:50.048] [E/2] [info] [Task Finished] Transform to mem in 0.259129s (14246 normal rows, 0 exceptions)
[2021-03-31 18:13:50.050] [E/4] [info] [Task Finished] Transform to mem in 0.258840s (14233 normal rows, 0 exceptions)
[2021-03-31 18:13:50.107] [E/12] [info] [Task Finished] Transform to mem in 0.307634s (16320 normal rows, 0 exceptions)
[2021-03-31 18:13:50.107] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 2.64599s
[2021-03-31 18:13:50.107] [global] [info] [Transform Stage] Stage 0 total wall clock time: 40.932s, 54108450 input rows, time to process 1 row via fast path: 0.000756481ms

[2021-03-31 18:13:50.150] [global] [info] Writing 2264570 rows as output to file (344.74 MB)
[2021-03-31 18:13:50.175] [E/12] [info] [Task Finished] write to file in 0.024459s (87192 rows, 13.27 MB)
[2021-03-31 18:13:50.190] [E/14] [info] [Task Finished] write to file in 0.039986s (155541 rows, 23.68 MB)
[2021-03-31 18:13:50.191] [E/15] [info] [Task Finished] write to file in 0.040558s (155417 rows, 23.66 MB)
[2021-03-31 18:13:50.191] [E/5] [info] [Task Finished] write to file in 0.040921s (155541 rows, 23.68 MB)
[2021-03-31 18:13:50.191] [E/10] [info] [Task Finished] write to file in 0.041164s (155626 rows, 23.69 MB)
[2021-03-31 18:13:50.191] [E/3] [info] [Task Finished] write to file in 0.041232s (155476 rows, 23.67 MB)
[2021-03-31 18:13:50.192] [E/2] [info] [Task Finished] write to file in 0.041641s (155417 rows, 23.66 MB)
[2021-03-31 18:13:50.192] [E/13] [info] [Task Finished] write to file in 0.041688s (155606 rows, 23.69 MB)
[2021-03-31 18:13:50.192] [E/8] [info] [Task Finished] write to file in 0.041706s (155395 rows, 23.66 MB)
[2021-03-31 18:13:50.192] [E/4] [info] [Task Finished] write to file in 0.042002s (155476 rows, 23.67 MB)
[2021-03-31 18:13:50.192] [E/9] [info] [Task Finished] write to file in 0.042005s (155415 rows, 23.66 MB)
[2021-03-31 18:13:50.192] [E/6] [info] [Task Finished] write to file in 0.042068s (155530 rows, 23.67 MB)
[2021-03-31 18:13:50.193] [E/11] [info] [Task Finished] write to file in 0.042279s (155704 rows, 23.70 MB)
[2021-03-31 18:13:50.193] [driver] [info] [Task Finished] write to file in 0.042352s (155530 rows, 23.67 MB)
[2021-03-31 18:13:50.200] [E/1] [info] [Task Finished] write to file in 0.049275s (155704 rows, 23.70 MB)
[2021-03-31 18:13:50.200] [global] [info] writing output took 0.092606s
[2021-03-31 18:13:50.200] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.0926855s
[2021-03-31 18:13:50.200] [global] [info] [Transform Stage] Stage 0 took 2.94555s
[2021-03-31 18:13:50.200] [global] [info] Query Execution took 3.00356s. (planning: 0.057977s, execution: 2.94558s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.1.7
            |_|
    
using Python 3.6.9 (default, Mar 31 2021, 05:47:10) 
[GCC 10.2.0] on linux
>>> running tuplex on ['/data/zillow/Z2_preprocessed/zillow_Z2_10G.csv']
Tuplex startup time: 0.006013154983520508
Tuplex job time: 3.2742981910705566 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 15, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '10G', 'tuplex.env.hostname': '6bcf8f6a24af', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.optimizer.mergeExceptionsInOrder': 'true', 'tuplex.optimizer.operatorReordering': 'false', 'tuplex.optionalThreshold': '0.7', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.1.7-py3.6.egg/tuplex/libexec/tuplex_runtime', 'tuplex.runTimeMemory': '8MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp', 'tuplex.webui.mongodb.path': '/data/tuplex_history', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.0203724,"llvm_optimization_time_s":0.137375,"llvm_compilation_time_s":0.0619736,"total_compilation_time_s":0.206821,"sampling_time_s":0.110449,"stages":[{"stage_no":0,"fast_path_wall_time_s":40.932,"fast_path_time_s":2.646,"fast_path_per_row_time_ns":756.481,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0}]}
{"startupTime": 0.006013154983520508, "jobTime": 3.2742981910705566}
[2021-03-31 18:13:50.225] [local execution engine] [info] Stopped local executor E/1
[2021-03-31 18:13:50.226] [local execution engine] [info] Stopped local executor E/2
[2021-03-31 18:13:50.227] [local execution engine] [info] Stopped local executor E/3
[2021-03-31 18:13:50.228] [local execution engine] [info] Stopped local executor E/4
[2021-03-31 18:13:50.230] [local execution engine] [info] Stopped local executor E/5
[2021-03-31 18:13:50.231] [local execution engine] [info] Stopped local executor E/6
[2021-03-31 18:13:50.232] [local execution engine] [info] Stopped local executor E/7
[2021-03-31 18:13:50.233] [local execution engine] [info] Stopped local executor E/8
[2021-03-31 18:13:50.235] [local execution engine] [info] Stopped local executor E/9
[2021-03-31 18:13:50.236] [local execution engine] [info] Stopped local executor E/10
[2021-03-31 18:13:50.237] [local execution engine] [info] Stopped local executor E/11
[2021-03-31 18:13:50.238] [local execution engine] [info] Stopped local executor E/12
[2021-03-31 18:13:50.239] [local execution engine] [info] Stopped local executor E/13
[2021-03-31 18:13:50.241] [local execution engine] [info] Stopped local executor E/14
[2021-03-31 18:13:50.242] [local execution engine] [info] Stopped local executor E/15
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
