[2022-02-20 01:39:56.828] [local ee] [info] loaded runtime library from/usr/local/lib/python3.6/site-packages/Tuplex-0.3.1-py3.6.egg/tuplex/libexec/tuplex_runtime.so
[2022-02-20 01:39:56.828] [local ee] [info] initializing LLVM backend
[2022-02-20 01:39:56.828] [local ee] [warning] init JIT compiler also only in local mode
[2022-02-20 01:39:56.828] [LLVM] [info] compiling code for skylake-avx512
[2022-02-20 01:39:56.829] [memory] [info] allocated bitmap managed memory region (32.00 GB, 32.00 MB block size)
[2022-02-20 01:39:56.829] [local execution engine] [info] started driver (32.00 GB, 32.00 MB default partition size)
[2022-02-20 01:39:56.829] [fileinputoperator] [info] found 1 file (10.01 GB) to process.
[2022-02-20 01:39:56.829] [global] [info] sampled file:///data/zillow/Z2_preprocessed/zillow_Z2_10G.csv on 256.00 KB
[2022-02-20 01:39:57.252] [logical planner] [info] logical optimization took 0.020179ms
[2022-02-20 01:39:57.252] [codegen] [info] generating pipeline for (Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str]) -> (str) (13 operators pipelined)
outSchema is different than last operator's schema:
outSchema: (str)
last Op: (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2022-02-20 01:39:57.253] [codegen] [info] generating function extractBd for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str])) -> i64
[2022-02-20 01:39:57.254] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> boolean
[2022-02-20 01:39:57.254] [codegen] [info] generating function extractType for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64)) -> str
[2022-02-20 01:39:57.255] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> boolean
[2022-02-20 01:39:57.255] [codegen] [info] generating lambda function for ((Option[str],Option[str],Option[str],Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str)) -> str
[2022-02-20 01:39:57.256] [codegen] [info] generating lambda function for (Option[str]) -> str
[2022-02-20 01:39:57.257] [codegen] [info] generating function extractBa for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str)) -> f64
[2022-02-20 01:39:57.258] [codegen] [info] generating function extractSqft for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64)) -> i64
[2022-02-20 01:39:57.259] [codegen] [info] generating function extractOffer for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64)) -> str
[2022-02-20 01:39:57.260] [codegen] [info] generating function extractPrice for ((Option[str],Option[str],str,Option[str],Option[f64],Option[str],Option[str],Option[str],i64,str,str,f64,i64,str)) -> i64
[2022-02-20 01:39:57.261] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2022-02-20 01:39:57.261] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> boolean
[2022-02-20 01:39:57.262] [codegen] [info] generating lambda function for ((Option[str],Option[str],str,Option[str],Option[f64],i64,Option[str],Option[str],i64,str,str,f64,i64,str)) -> (Option[str],str,Option[str],str,Option[str],i64,f64,i64,str,str,i64)
[2022-02-20 01:39:57.298] [global] [info] retrieved metrics object
[2022-02-20 01:39:57.434] [global] [info] Optimization via LLVM passes took 0.138482 ms
[2022-02-20 01:39:57.434] [global] [info] starting code compilation
[2022-02-20 01:39:57.434] [global] [info] first compile done
[2022-02-20 01:39:57.497] [global] [info] functor Stage_0 retrieved from llvm
[2022-02-20 01:39:57.497] [global] [info] retrieving init/release stage functors
[2022-02-20 01:39:57.497] [global] [info] Compiled code paths for stage 0 in 0.06 ms
[2022-02-20 01:39:57.497] [global] [info] [Transform Stage] Stage 0 compiled to x86 in 0.208899s
[2022-02-20 01:39:57.498] [local ee] [info] split /data/zillow/Z2_preprocessed/zillow_Z2_10G.csv into 160 tasks
[2022-02-20 01:39:57.761] [driver] [info] [Task Finished] Transform to mem in 0.263512s (14226 normal rows, 0 exceptions)
[2022-02-20 01:39:58.018] [driver] [info] [Task Finished] Transform to mem in 0.256768s (14206 normal rows, 0 exceptions)
[2022-02-20 01:39:58.274] [driver] [info] [Task Finished] Transform to mem in 0.256517s (13970 normal rows, 0 exceptions)
[2022-02-20 01:39:58.532] [driver] [info] [Task Finished] Transform to mem in 0.257086s (14253 normal rows, 0 exceptions)
[2022-02-20 01:39:58.789] [driver] [info] [Task Finished] Transform to mem in 0.256954s (14217 normal rows, 0 exceptions)
[2022-02-20 01:39:59.045] [driver] [info] [Task Finished] Transform to mem in 0.256013s (13920 normal rows, 0 exceptions)
[2022-02-20 01:39:59.302] [driver] [info] [Task Finished] Transform to mem in 0.257587s (14267 normal rows, 0 exceptions)
[2022-02-20 01:39:59.559] [driver] [info] [Task Finished] Transform to mem in 0.257276s (14248 normal rows, 0 exceptions)
[2022-02-20 01:39:59.815] [driver] [info] [Task Finished] Transform to mem in 0.255931s (13953 normal rows, 0 exceptions)
[2022-02-20 01:40:00.073] [driver] [info] [Task Finished] Transform to mem in 0.257144s (14207 normal rows, 0 exceptions)
[2022-02-20 01:40:00.329] [driver] [info] [Task Finished] Transform to mem in 0.256854s (14237 normal rows, 0 exceptions)
[2022-02-20 01:40:00.586] [driver] [info] [Task Finished] Transform to mem in 0.256400s (13988 normal rows, 0 exceptions)
[2022-02-20 01:40:00.843] [driver] [info] [Task Finished] Transform to mem in 0.257257s (14177 normal rows, 0 exceptions)
[2022-02-20 01:40:01.103] [driver] [info] [Task Finished] Transform to mem in 0.259846s (14242 normal rows, 0 exceptions)
[2022-02-20 01:40:01.359] [driver] [info] [Task Finished] Transform to mem in 0.256093s (14020 normal rows, 0 exceptions)
[2022-02-20 01:40:01.616] [driver] [info] [Task Finished] Transform to mem in 0.256747s (14162 normal rows, 0 exceptions)
[2022-02-20 01:40:01.872] [driver] [info] [Task Finished] Transform to mem in 0.255637s (14241 normal rows, 0 exceptions)
[2022-02-20 01:40:02.126] [driver] [info] [Task Finished] Transform to mem in 0.254836s (14059 normal rows, 0 exceptions)
[2022-02-20 01:40:02.382] [driver] [info] [Task Finished] Transform to mem in 0.255788s (14114 normal rows, 0 exceptions)
[2022-02-20 01:40:02.638] [driver] [info] [Task Finished] Transform to mem in 0.255699s (14238 normal rows, 0 exceptions)
[2022-02-20 01:40:02.893] [driver] [info] [Task Finished] Transform to mem in 0.254931s (14081 normal rows, 0 exceptions)
[2022-02-20 01:40:03.149] [driver] [info] [Task Finished] Transform to mem in 0.255918s (14095 normal rows, 0 exceptions)
[2022-02-20 01:40:03.404] [driver] [info] [Task Finished] Transform to mem in 0.255490s (14221 normal rows, 0 exceptions)
[2022-02-20 01:40:03.659] [driver] [info] [Task Finished] Transform to mem in 0.254949s (14086 normal rows, 0 exceptions)
[2022-02-20 01:40:03.915] [driver] [info] [Task Finished] Transform to mem in 0.255331s (14097 normal rows, 0 exceptions)
[2022-02-20 01:40:04.171] [driver] [info] [Task Finished] Transform to mem in 0.255895s (14212 normal rows, 0 exceptions)
[2022-02-20 01:40:04.425] [driver] [info] [Task Finished] Transform to mem in 0.254910s (14108 normal rows, 0 exceptions)
[2022-02-20 01:40:04.684] [driver] [info] [Task Finished] Transform to mem in 0.258909s (14100 normal rows, 0 exceptions)
[2022-02-20 01:40:04.940] [driver] [info] [Task Finished] Transform to mem in 0.255983s (14212 normal rows, 0 exceptions)
[2022-02-20 01:40:05.195] [driver] [info] [Task Finished] Transform to mem in 0.254741s (14099 normal rows, 0 exceptions)
[2022-02-20 01:40:05.450] [driver] [info] [Task Finished] Transform to mem in 0.255268s (14094 normal rows, 0 exceptions)
[2022-02-20 01:40:05.706] [driver] [info] [Task Finished] Transform to mem in 0.255312s (14223 normal rows, 0 exceptions)
[2022-02-20 01:40:05.961] [driver] [info] [Task Finished] Transform to mem in 0.254941s (14089 normal rows, 0 exceptions)
[2022-02-20 01:40:06.216] [driver] [info] [Task Finished] Transform to mem in 0.255393s (14083 normal rows, 0 exceptions)
[2022-02-20 01:40:06.472] [driver] [info] [Task Finished] Transform to mem in 0.255794s (14235 normal rows, 0 exceptions)
[2022-02-20 01:40:06.727] [driver] [info] [Task Finished] Transform to mem in 0.254979s (14170 normal rows, 0 exceptions)
[2022-02-20 01:40:06.982] [driver] [info] [Task Finished] Transform to mem in 0.255230s (14002 normal rows, 0 exceptions)
[2022-02-20 01:40:07.238] [driver] [info] [Task Finished] Transform to mem in 0.255888s (14226 normal rows, 0 exceptions)
[2022-02-20 01:40:07.493] [driver] [info] [Task Finished] Transform to mem in 0.255292s (14206 normal rows, 0 exceptions)
[2022-02-20 01:40:07.749] [driver] [info] [Task Finished] Transform to mem in 0.255448s (13961 normal rows, 0 exceptions)
[2022-02-20 01:40:08.004] [driver] [info] [Task Finished] Transform to mem in 0.255568s (14246 normal rows, 0 exceptions)
[2022-02-20 01:40:08.262] [driver] [info] [Task Finished] Transform to mem in 0.257934s (14233 normal rows, 0 exceptions)
[2022-02-20 01:40:08.517] [driver] [info] [Task Finished] Transform to mem in 0.254999s (13915 normal rows, 0 exceptions)
[2022-02-20 01:40:08.773] [driver] [info] [Task Finished] Transform to mem in 0.255662s (14253 normal rows, 0 exceptions)
[2022-02-20 01:40:09.029] [driver] [info] [Task Finished] Transform to mem in 0.255387s (14267 normal rows, 0 exceptions)
[2022-02-20 01:40:09.283] [driver] [info] [Task Finished] Transform to mem in 0.254469s (13892 normal rows, 0 exceptions)
[2022-02-20 01:40:09.539] [driver] [info] [Task Finished] Transform to mem in 0.255869s (14245 normal rows, 0 exceptions)
[2022-02-20 01:40:09.794] [driver] [info] [Task Finished] Transform to mem in 0.255388s (14260 normal rows, 0 exceptions)
[2022-02-20 01:40:10.049] [driver] [info] [Task Finished] Transform to mem in 0.254855s (13978 normal rows, 0 exceptions)
[2022-02-20 01:40:10.305] [driver] [info] [Task Finished] Transform to mem in 0.255361s (14181 normal rows, 0 exceptions)
[2022-02-20 01:40:10.560] [driver] [info] [Task Finished] Transform to mem in 0.255515s (14234 normal rows, 0 exceptions)
[2022-02-20 01:40:10.815] [driver] [info] [Task Finished] Transform to mem in 0.254880s (14015 normal rows, 0 exceptions)
[2022-02-20 01:40:11.071] [driver] [info] [Task Finished] Transform to mem in 0.255599s (14142 normal rows, 0 exceptions)
[2022-02-20 01:40:11.326] [driver] [info] [Task Finished] Transform to mem in 0.255367s (14280 normal rows, 0 exceptions)
[2022-02-20 01:40:11.581] [driver] [info] [Task Finished] Transform to mem in 0.255035s (13982 normal rows, 0 exceptions)
[2022-02-20 01:40:11.837] [driver] [info] [Task Finished] Transform to mem in 0.255881s (14173 normal rows, 0 exceptions)
[2022-02-20 01:40:12.096] [driver] [info] [Task Finished] Transform to mem in 0.258647s (14235 normal rows, 0 exceptions)
[2022-02-20 01:40:12.351] [driver] [info] [Task Finished] Transform to mem in 0.255154s (14054 normal rows, 0 exceptions)
[2022-02-20 01:40:12.607] [driver] [info] [Task Finished] Transform to mem in 0.256252s (14129 normal rows, 0 exceptions)
[2022-02-20 01:40:12.862] [driver] [info] [Task Finished] Transform to mem in 0.255307s (14223 normal rows, 0 exceptions)
[2022-02-20 01:40:13.118] [driver] [info] [Task Finished] Transform to mem in 0.255864s (14081 normal rows, 0 exceptions)
[2022-02-20 01:40:13.377] [driver] [info] [Task Finished] Transform to mem in 0.258563s (14108 normal rows, 0 exceptions)
[2022-02-20 01:40:13.632] [driver] [info] [Task Finished] Transform to mem in 0.255425s (14208 normal rows, 0 exceptions)
[2022-02-20 01:40:13.887] [driver] [info] [Task Finished] Transform to mem in 0.254941s (14089 normal rows, 0 exceptions)
[2022-02-20 01:40:14.142] [driver] [info] [Task Finished] Transform to mem in 0.254862s (14094 normal rows, 0 exceptions)
[2022-02-20 01:40:14.397] [driver] [info] [Task Finished] Transform to mem in 0.255164s (14232 normal rows, 0 exceptions)
[2022-02-20 01:40:14.652] [driver] [info] [Task Finished] Transform to mem in 0.255015s (14088 normal rows, 0 exceptions)
[2022-02-20 01:40:14.907] [driver] [info] [Task Finished] Transform to mem in 0.254987s (14106 normal rows, 0 exceptions)
[2022-02-20 01:40:15.163] [driver] [info] [Task Finished] Transform to mem in 0.255267s (14211 normal rows, 0 exceptions)
[2022-02-20 01:40:15.417] [driver] [info] [Task Finished] Transform to mem in 0.254362s (14113 normal rows, 0 exceptions)
[2022-02-20 01:40:15.675] [driver] [info] [Task Finished] Transform to mem in 0.258347s (14075 normal rows, 0 exceptions)
[2022-02-20 01:40:15.931] [driver] [info] [Task Finished] Transform to mem in 0.255607s (14223 normal rows, 0 exceptions)
[2022-02-20 01:40:16.190] [driver] [info] [Task Finished] Transform to mem in 0.258677s (14095 normal rows, 0 exceptions)
[2022-02-20 01:40:16.444] [driver] [info] [Task Finished] Transform to mem in 0.254768s (14086 normal rows, 0 exceptions)
[2022-02-20 01:40:16.700] [driver] [info] [Task Finished] Transform to mem in 0.255554s (14226 normal rows, 0 exceptions)
[2022-02-20 01:40:16.958] [driver] [info] [Task Finished] Transform to mem in 0.258197s (14170 normal rows, 0 exceptions)
[2022-02-20 01:40:17.213] [driver] [info] [Task Finished] Transform to mem in 0.254724s (14002 normal rows, 0 exceptions)
[2022-02-20 01:40:17.468] [driver] [info] [Task Finished] Transform to mem in 0.255532s (14226 normal rows, 0 exceptions)
[2022-02-20 01:40:17.724] [driver] [info] [Task Finished] Transform to mem in 0.255005s (14206 normal rows, 0 exceptions)
[2022-02-20 01:40:17.979] [driver] [info] [Task Finished] Transform to mem in 0.255310s (13970 normal rows, 0 exceptions)
[2022-02-20 01:40:18.234] [driver] [info] [Task Finished] Transform to mem in 0.255087s (14237 normal rows, 0 exceptions)
[2022-02-20 01:40:18.489] [driver] [info] [Task Finished] Transform to mem in 0.255013s (14233 normal rows, 0 exceptions)
[2022-02-20 01:40:18.743] [driver] [info] [Task Finished] Transform to mem in 0.254222s (13920 normal rows, 0 exceptions)
[2022-02-20 01:40:18.999] [driver] [info] [Task Finished] Transform to mem in 0.255531s (14267 normal rows, 0 exceptions)
[2022-02-20 01:40:19.253] [driver] [info] [Task Finished] Transform to mem in 0.254533s (14248 normal rows, 0 exceptions)
[2022-02-20 01:40:19.510] [driver] [info] [Task Finished] Transform to mem in 0.257038s (13912 normal rows, 0 exceptions)
[2022-02-20 01:40:19.769] [driver] [info] [Task Finished] Transform to mem in 0.258558s (14248 normal rows, 0 exceptions)
[2022-02-20 01:40:20.023] [driver] [info] [Task Finished] Transform to mem in 0.254424s (14237 normal rows, 0 exceptions)
[2022-02-20 01:40:20.278] [driver] [info] [Task Finished] Transform to mem in 0.254172s (13988 normal rows, 0 exceptions)
[2022-02-20 01:40:20.536] [driver] [info] [Task Finished] Transform to mem in 0.258894s (14171 normal rows, 0 exceptions)
[2022-02-20 01:40:20.791] [driver] [info] [Task Finished] Transform to mem in 0.254994s (14234 normal rows, 0 exceptions)
[2022-02-20 01:40:21.046] [driver] [info] [Task Finished] Transform to mem in 0.254293s (14015 normal rows, 0 exceptions)
[2022-02-20 01:40:21.301] [driver] [info] [Task Finished] Transform to mem in 0.255252s (14176 normal rows, 0 exceptions)
[2022-02-20 01:40:21.556] [driver] [info] [Task Finished] Transform to mem in 0.254784s (14246 normal rows, 0 exceptions)
[2022-02-20 01:40:21.811] [driver] [info] [Task Finished] Transform to mem in 0.255072s (14059 normal rows, 0 exceptions)
[2022-02-20 01:40:22.066] [driver] [info] [Task Finished] Transform to mem in 0.254915s (14114 normal rows, 0 exceptions)
[2022-02-20 01:40:22.321] [driver] [info] [Task Finished] Transform to mem in 0.255103s (14238 normal rows, 0 exceptions)
[2022-02-20 01:40:22.576] [driver] [info] [Task Finished] Transform to mem in 0.254937s (14081 normal rows, 0 exceptions)
[2022-02-20 01:40:22.831] [driver] [info] [Task Finished] Transform to mem in 0.255134s (14095 normal rows, 0 exceptions)
[2022-02-20 01:40:23.089] [driver] [info] [Task Finished] Transform to mem in 0.257867s (14221 normal rows, 0 exceptions)
[2022-02-20 01:40:23.344] [driver] [info] [Task Finished] Transform to mem in 0.254554s (14086 normal rows, 0 exceptions)
[2022-02-20 01:40:23.602] [driver] [info] [Task Finished] Transform to mem in 0.258088s (14091 normal rows, 0 exceptions)
[2022-02-20 01:40:23.859] [driver] [info] [Task Finished] Transform to mem in 0.257117s (14218 normal rows, 0 exceptions)
[2022-02-20 01:40:24.113] [driver] [info] [Task Finished] Transform to mem in 0.254627s (14095 normal rows, 0 exceptions)
[2022-02-20 01:40:24.372] [driver] [info] [Task Finished] Transform to mem in 0.258207s (14113 normal rows, 0 exceptions)
[2022-02-20 01:40:24.627] [driver] [info] [Task Finished] Transform to mem in 0.255369s (14197 normal rows, 0 exceptions)
[2022-02-20 01:40:24.882] [driver] [info] [Task Finished] Transform to mem in 0.254682s (14114 normal rows, 0 exceptions)
[2022-02-20 01:40:25.136] [driver] [info] [Task Finished] Transform to mem in 0.254553s (14094 normal rows, 0 exceptions)
[2022-02-20 01:40:25.391] [driver] [info] [Task Finished] Transform to mem in 0.255111s (14197 normal rows, 0 exceptions)
[2022-02-20 01:40:25.646] [driver] [info] [Task Finished] Transform to mem in 0.254548s (14115 normal rows, 0 exceptions)
[2022-02-20 01:40:25.901] [driver] [info] [Task Finished] Transform to mem in 0.254660s (14083 normal rows, 0 exceptions)
[2022-02-20 01:40:26.156] [driver] [info] [Task Finished] Transform to mem in 0.255455s (14235 normal rows, 0 exceptions)
[2022-02-20 01:40:26.411] [driver] [info] [Task Finished] Transform to mem in 0.254852s (14144 normal rows, 0 exceptions)
[2022-02-20 01:40:26.668] [driver] [info] [Task Finished] Transform to mem in 0.257442s (14015 normal rows, 0 exceptions)
[2022-02-20 01:40:26.924] [driver] [info] [Task Finished] Transform to mem in 0.255417s (14237 normal rows, 0 exceptions)
[2022-02-20 01:40:27.182] [driver] [info] [Task Finished] Transform to mem in 0.257782s (14159 normal rows, 0 exceptions)
[2022-02-20 01:40:27.436] [driver] [info] [Task Finished] Transform to mem in 0.254181s (14010 normal rows, 0 exceptions)
[2022-02-20 01:40:27.691] [driver] [info] [Task Finished] Transform to mem in 0.255406s (14246 normal rows, 0 exceptions)
[2022-02-20 01:40:27.950] [driver] [info] [Task Finished] Transform to mem in 0.258352s (14178 normal rows, 0 exceptions)
[2022-02-20 01:40:28.204] [driver] [info] [Task Finished] Transform to mem in 0.254113s (13970 normal rows, 0 exceptions)
[2022-02-20 01:40:28.460] [driver] [info] [Task Finished] Transform to mem in 0.255682s (14253 normal rows, 0 exceptions)
[2022-02-20 01:40:28.714] [driver] [info] [Task Finished] Transform to mem in 0.254618s (14236 normal rows, 0 exceptions)
[2022-02-20 01:40:28.969] [driver] [info] [Task Finished] Transform to mem in 0.254530s (13901 normal rows, 0 exceptions)
[2022-02-20 01:40:29.224] [driver] [info] [Task Finished] Transform to mem in 0.255258s (14267 normal rows, 0 exceptions)
[2022-02-20 01:40:29.479] [driver] [info] [Task Finished] Transform to mem in 0.254821s (14248 normal rows, 0 exceptions)
[2022-02-20 01:40:29.733] [driver] [info] [Task Finished] Transform to mem in 0.254607s (13953 normal rows, 0 exceptions)
[2022-02-20 01:40:29.989] [driver] [info] [Task Finished] Transform to mem in 0.255701s (14207 normal rows, 0 exceptions)
[2022-02-20 01:40:30.248] [driver] [info] [Task Finished] Transform to mem in 0.258363s (14245 normal rows, 0 exceptions)
[2022-02-20 01:40:30.506] [driver] [info] [Task Finished] Transform to mem in 0.258743s (13999 normal rows, 0 exceptions)
[2022-02-20 01:40:30.762] [driver] [info] [Task Finished] Transform to mem in 0.256128s (14158 normal rows, 0 exceptions)
[2022-02-20 01:40:31.020] [driver] [info] [Task Finished] Transform to mem in 0.257581s (14242 normal rows, 0 exceptions)
[2022-02-20 01:40:31.275] [driver] [info] [Task Finished] Transform to mem in 0.254766s (14020 normal rows, 0 exceptions)
[2022-02-20 01:40:31.531] [driver] [info] [Task Finished] Transform to mem in 0.255819s (14173 normal rows, 0 exceptions)
[2022-02-20 01:40:31.786] [driver] [info] [Task Finished] Transform to mem in 0.255134s (14235 normal rows, 0 exceptions)
[2022-02-20 01:40:32.041] [driver] [info] [Task Finished] Transform to mem in 0.255356s (14054 normal rows, 0 exceptions)
[2022-02-20 01:40:32.296] [driver] [info] [Task Finished] Transform to mem in 0.254980s (14114 normal rows, 0 exceptions)
[2022-02-20 01:40:32.551] [driver] [info] [Task Finished] Transform to mem in 0.255168s (14238 normal rows, 0 exceptions)
[2022-02-20 01:40:32.806] [driver] [info] [Task Finished] Transform to mem in 0.254452s (14081 normal rows, 0 exceptions)
[2022-02-20 01:40:33.061] [driver] [info] [Task Finished] Transform to mem in 0.255142s (14095 normal rows, 0 exceptions)
[2022-02-20 01:40:33.316] [driver] [info] [Task Finished] Transform to mem in 0.254823s (14221 normal rows, 0 exceptions)
[2022-02-20 01:40:33.571] [driver] [info] [Task Finished] Transform to mem in 0.254703s (14089 normal rows, 0 exceptions)
[2022-02-20 01:40:33.825] [driver] [info] [Task Finished] Transform to mem in 0.254514s (14094 normal rows, 0 exceptions)
[2022-02-20 01:40:34.083] [driver] [info] [Task Finished] Transform to mem in 0.258103s (14212 normal rows, 0 exceptions)
[2022-02-20 01:40:34.337] [driver] [info] [Task Finished] Transform to mem in 0.254075s (14108 normal rows, 0 exceptions)
[2022-02-20 01:40:34.595] [driver] [info] [Task Finished] Transform to mem in 0.257849s (14100 normal rows, 0 exceptions)
[2022-02-20 01:40:34.850] [driver] [info] [Task Finished] Transform to mem in 0.254791s (14217 normal rows, 0 exceptions)
[2022-02-20 01:40:35.104] [driver] [info] [Task Finished] Transform to mem in 0.254260s (14094 normal rows, 0 exceptions)
[2022-02-20 01:40:35.359] [driver] [info] [Task Finished] Transform to mem in 0.254773s (14094 normal rows, 0 exceptions)
[2022-02-20 01:40:35.614] [driver] [info] [Task Finished] Transform to mem in 0.254957s (14223 normal rows, 0 exceptions)
[2022-02-20 01:40:35.868] [driver] [info] [Task Finished] Transform to mem in 0.254381s (14089 normal rows, 0 exceptions)
[2022-02-20 01:40:36.123] [driver] [info] [Task Finished] Transform to mem in 0.254480s (14086 normal rows, 0 exceptions)
[2022-02-20 01:40:36.378] [driver] [info] [Task Finished] Transform to mem in 0.255224s (14232 normal rows, 0 exceptions)
[2022-02-20 01:40:36.633] [driver] [info] [Task Finished] Transform to mem in 0.254623s (14170 normal rows, 0 exceptions)
[2022-02-20 01:40:36.887] [driver] [info] [Task Finished] Transform to mem in 0.254158s (14002 normal rows, 0 exceptions)
[2022-02-20 01:40:37.142] [driver] [info] [Task Finished] Transform to mem in 0.255303s (14226 normal rows, 0 exceptions)
[2022-02-20 01:40:37.397] [driver] [info] [Task Finished] Transform to mem in 0.254186s (14206 normal rows, 0 exceptions)
[2022-02-20 01:40:37.650] [driver] [info] [Task Finished] Transform to mem in 0.253932s (13961 normal rows, 0 exceptions)
[2022-02-20 01:40:37.908] [driver] [info] [Task Finished] Transform to mem in 0.258002s (14246 normal rows, 0 exceptions)
[2022-02-20 01:40:38.166] [driver] [info] [Task Finished] Transform to mem in 0.257646s (14233 normal rows, 0 exceptions)
[2022-02-20 01:40:38.462] [driver] [info] [Task Finished] Transform to mem in 0.296271s (16320 normal rows, 0 exceptions)
[2022-02-20 01:40:38.462] [global] [info] [Transform Stage] Stage 0 completed 160 load&transform tasks in 40.9653s
[2022-02-20 01:40:38.462] [global] [info] [Transform Stage] Stage 0 total wall clock time: 40.9621s, 54108450 input rows, time to process 1 row via fast path: 0.000757037ms

[2022-02-20 01:40:38.463] [global] [info] Writing 2264570 rows as output to file (344.74 MB)
[2022-02-20 01:40:38.863] [driver] [info] [Task Finished] write to file in 0.400315s (2264570 rows, 344.74 MB)
[2022-02-20 01:40:38.863] [global] [info] writing output took 0.400714s
[2022-02-20 01:40:38.863] [global] [info] [Transform Stage] Stage 0 completed 160 sink tasks in 0.400815s
[2022-02-20 01:40:38.864] [global] [info] [Transform Stage] Stage 0 took 41.5753s
[2022-02-20 01:40:38.864] [global] [info] Query Execution took 41.6339s. (planning: 0.0585337s, execution: 41.5753s)
Welcome to

  _____            _
 |_   _|   _ _ __ | | _____  __
   | || | | | '_ \| |/ _ \ \/ /
   | || |_| | |_) | |  __/>  <
   |_| \__,_| .__/|_|\___/_/\_\ 0.3.1
            |_|
    
using Python 3.6.9 (default, Feb  8 2022, 16:30:43) 
[GCC 9.3.0] on linux
>>> running tuplex on ['/data/zillow/Z2_preprocessed/zillow_Z2_10G.csv']
Tuplex startup time: 0.007927656173706055
Tuplex job time: 42.03598237037659 s
{'tuplex.useLLVMOptimizer': True, 'tuplex.autoUpcast': False, 'tuplex.allowUndefinedBehavior': False, 'tuplex.optimizer.codeStats': False, 'tuplex.optimizer.generateParser': False, 'tuplex.optimizer.nullValueOptimization': False, 'tuplex.optimizer.filterPushdown': True, 'tuplex.optimizer.sharedObjectPropagation': True, 'tuplex.optimizer.mergeExceptionsInOrder': True, 'tuplex.optimizer.operatorReordering': False, 'tuplex.interleaveIO': True, 'tuplex.resolveWithInterpreterOnly': False, 'tuplex.network.verifySSL': False, 'tuplex.redirectToPythonLogging': False, 'tuplex.csv.selectionPushdown': True, 'tuplex.webui.enable': False, 'tuplex.executorCount': 0, 'tuplex.csv.maxDetectionRows': 10000, 'tuplex.webui.port': 5000, 'tuplex.webui.mongodb.port': 27017, 'tuplex.webui.exceptionDisplayLimit': 5, 'tuplex.normalcaseThreshold': 0.9, 'tuplex.optionalThreshold': 0.7, 'tuplex.aws.connectTimeout': '1', 'tuplex.aws.httpThreadCount': '32', 'tuplex.aws.lambdaMemory': '1536', 'tuplex.aws.lambdaTimeout': '600', 'tuplex.aws.maxConcurrency': '100', 'tuplex.aws.region': 'us-east-1', 'tuplex.aws.requestTimeout': '600', 'tuplex.aws.requesterPay': 'false', 'tuplex.aws.scratchDir': '', 'tuplex.backend': 'local', 'tuplex.csv.comments': ['#', '~'], 'tuplex.csv.maxDetectionMemory': '256KB', 'tuplex.csv.quotechar': '"', 'tuplex.csv.separators': [',', ';', '|', '\t'], 'tuplex.driverMemory': '32G', 'tuplex.env.hostname': 'cc956b40c9fa', 'tuplex.env.mode': 'file', 'tuplex.env.user': 'root', 'tuplex.executorMemory': '6G', 'tuplex.inputSplitSize': '64MB', 'tuplex.logDir': '.', 'tuplex.network.caFile': '', 'tuplex.network.caPath': '', 'tuplex.partitionSize': '32MB', 'tuplex.readBufferSize': '128KB', 'tuplex.runTimeLibrary': '/usr/local/lib/python3.6/site-packages/Tuplex-0.3.1-py3.6.egg/tuplex/libexec/tuplex_runtime.so', 'tuplex.runTimeMemory': '8MB', 'tuplex.runTimeMemoryBlockSize': '4MB', 'tuplex.scratchDir': '/tmp/tuplex-cache-tuplex', 'tuplex.webui.mongodb.path': '/tmp/tuplex-cache-tuplex/mongodb', 'tuplex.webui.mongodb.url': 'localhost', 'tuplex.webui.url': 'localhost'}
{"logical_optimization_time_s":0.0201793,"llvm_optimization_time_s":0.138482,"llvm_compilation_time_s":0.0629609,"total_compilation_time_s":0.208897,"sampling_time_s":0.115151,"stages":[{"stage_no":0,"fast_path_wall_time_s":40.9621,"fast_path_time_s":40.9654,"fast_path_per_row_time_ns":757.037,"slow_path_wall_time_s":0,"slow_path_time_s":0,"slow_path_per_row_time_ns":0,"partitions_swapin_count":0,"partitions_swapout_count":0,"partitions_bytes_swapped_in":0,"partitions_bytes_swapped_out":0}]}
{"startupTime": 0.007927656173706055, "jobTime": 42.03598237037659}
[GLOBAL] Local engine terminated.
[GLOBAL] releasing 1 active partitions
