Commits

Branch selector

 master

User selector

sunchao

Datepicker

All time

Commit History

Commits on Apr 12, 2022

• [SPARK-34863][SQL][FOLLOWUP] Add benchmark for Parquet & ORC nested column scan

  Show description for 3640772
  sunchao

  committed
  3640772

  Copy full SHA for 3640772

Commits on Apr 9, 2022

• [SPARK-38840][SQL] Enable `spark.sql.parquet.enableNestedColumnVectorizedReader` on master by default

  Show description for 233dd98
  

  sunchao

  authored and

  viirya

  committed
  233dd98

  Copy full SHA for 233dd98

Commits on Apr 5, 2022

• [SPARK-37377][SQL] Initial implementation of Storage-Partitioned Join

  Show description for 20ffbf7
  

  sunchao

  authored and

  dongjoon-hyun

  committed
  20ffbf7

  Copy full SHA for 20ffbf7

Commits on Apr 4, 2022

• [SPARK-34863][SQL][FOLLOWUP] Disable `spark.sql.parquet.enableNestedColumnVectorizedReader` by default

  Show description for 1b08673
  

  sunchao

  authored and

  viirya

  committed
  1b08673

  Copy full SHA for 1b08673

Commits on Apr 2, 2022

• [SPARK-34863][SQL] Support complex types for Parquet vectorized reader

  Show description for deac8f9
  

  sunchao

  authored and

  viirya

  committed
  deac8f9

  Copy full SHA for deac8f9

Commits on Mar 9, 2022

• [SPARK-37600][BUILD] Upgrade to Hadoop 3.3.2

  Show description for 4da04fc
  

  sunchao

  authored and

  dongjoon-hyun

  committed
  4da04fc

  Copy full SHA for 4da04fc

Commits on Feb 10, 2022

• [SPARK-38134][BUILD] Upgrade Apache Arrow to 7.0.0

  Show description for 0ae8c39
  

  sunchao

  authored and

  dongjoon-hyun

  committed
  0ae8c39

  Copy full SHA for 0ae8c39

Commits on Jan 19, 2022

• [SPARK-37957][SQL] Correctly pass deterministic flag for V2 scalar functions

  Show description for 9ffca0c
  sunchao

  committed
  9ffca0c

  Copy full SHA for 9ffca0c

Commits on Dec 27, 2021

• [SPARK-35703][SQL] Relax constraint for bucket join and remove HashClusteredDistribution

  Show description for d8920ae
  

  sunchao

  authored and

  cloud-fan

  committed
  d8920ae

  Copy full SHA for d8920ae

Commits on Dec 8, 2021

• [SPARK-37205][YARN] Introduce a new config 'spark.yarn.am.tokenConfRegex' to support renewing delegation tokens in a multi-cluster environment

  Show description for 77a8778
  sunchao

  committed
  77a8778

  Copy full SHA for 77a8778

Commits on Dec 1, 2021

• [SPARK-37376][SQL] Introduce a new DataSource V2 interface HasPartitionKey

  Show description for e7fa289
  

  sunchao

  authored and

  dongjoon-hyun

  committed
  e7fa289

  Copy full SHA for e7fa289

Commits on Nov 16, 2021

• [SPARK-37342][BUILD] Upgrade Apache Arrow to 6.0.0

  Show description for 9665461
  

  sunchao

  authored and

  HyukjinKwon

  committed
  9665461

  Copy full SHA for 9665461

Commits on Nov 2, 2021

• [SPARK-36935][SQL] Extend ParquetSchemaConverter to compute Parquet repetition & definition level

  Show description for d246010
  

  sunchao

  authored and

  dongjoon-hyun

  committed
  d246010

  Copy full SHA for d246010

Commits on Oct 26, 2021

• [SPARK-37113][BUILD] Upgrade Parquet to 1.12.2

  Show description for 87591c9
  

  sunchao

  authored and

  dbtsai

  committed
  87591c9

  Copy full SHA for 87591c9

Commits on Oct 22, 2021

• [SPARK-37069][SQL] Properly fallback when Hive.getWithoutRegisterFns is not available

  Show description for 39a0c22
  

  sunchao

  authored and

  HyukjinKwon

  committed
  39a0c22

  Copy full SHA for 39a0c22

Commits on Oct 2, 2021

• [SPARK-36891][SQL] Refactor SpecificParquetRecordReaderBase and add more coverage on vectorized Parquet decoding

  Show description for 14d4cee
  

  sunchao

  authored and

  dongjoon-hyun

  committed
  14d4cee

  Copy full SHA for 14d4cee

Commits on Sep 28, 2021

• [SPARK-36873][BUILD][TEST-MAVEN] Add provided Guava dependency for network-yarn module

  Show description for 53f58b6
  

  sunchao

  authored and

  gengliangwang

  committed
  53f58b6

  Copy full SHA for 53f58b6

• [SPARK-36863][BUILD] Update dependency manifest files for all released Spark artifacts

  Show description for 8d8b710
  

  sunchao

  authored and

  HyukjinKwon

  committed
  8d8b710

  Copy full SHA for 8d8b710

Commits on Sep 27, 2021

• [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Move spark.yarn.isHadoopProvided to parent pom

  Show description for f9efdee
  

  sunchao

  authored and

  gengliangwang

  committed
  f9efdee

  Copy full SHA for f9efdee

Commits on Sep 26, 2021

• [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom

  Show description for 937a74e
  

  sunchao

  authored and

  gengliangwang

  committed
  937a74e

  Copy full SHA for 937a74e

Commits on Sep 24, 2021

• [SPARK-36835][BUILD] Enable createDependencyReducedPom for Maven shaded plugin

  Show description for ed88e61
  

  sunchao

  authored and

  gengliangwang

  committed
  ed88e61

  Copy full SHA for ed88e61

Commits on Sep 22, 2021

• [SPARK-36820][SQL] Disable tests related to LZ4 for Hadoop 2.7 profile

  Show description for 6eb7559
  

  sunchao

  authored and

  viirya

  committed
  6eb7559

  Copy full SHA for 6eb7559

Commits on Sep 15, 2021

• [SPARK-36726] Upgrade Parquet to 1.12.1

  Show description for a927b08
  

  sunchao

  authored and

  dbtsai

  committed
  a927b08

  Copy full SHA for a927b08

Commits on Aug 2, 2021

• [SPARK-36137][SQL] HiveShim should fallback to getAllPartitionsOf even if directSQL is enabled in remote HMS

  Show description for 7a27f8a
  

  sunchao

  authored and

  viirya

  committed
  7a27f8a

  Copy full SHA for 7a27f8a

Commits on Jul 30, 2021

• [SPARK-36136][SQL][TESTS] Refactor PruneFileSourcePartitionsSuite etc to a different package

  Show description for 0ece865
  

  sunchao

  authored and

  viirya

  committed
  0ece865

  Copy full SHA for 0ece865

Commits on Jul 26, 2021

• [SPARK-36136][SQL][TESTS] Refactor PruneFileSourcePartitionsSuite etc to a different package

  Show description for 634f96d
  

  sunchao

  authored and

  viirya

  committed
  634f96d

  Copy full SHA for 634f96d

Commits on Jul 16, 2021

• [SPARK-36128][SQL] Apply spark.sql.hive.metastorePartitionPruning for non-Hive tables that uses Hive metastore for partition management

  Show description for 37dc3f9
  

  sunchao

  authored and

  viirya

  committed
  37dc3f9

  Copy full SHA for 37dc3f9

Commits on Jul 14, 2021

• [SPARK-36123][SQL] Parquet vectorized reader doesn't skip null values correctly

  Show description for e980c7a
  

  sunchao

  authored and

  cloud-fan

  committed
  e980c7a

  Copy full SHA for e980c7a

• [SPARK-36131][SQL][TEST] Refactor ParquetColumnIndexSuite

  Show description for 7a7b086
  

  sunchao

  authored and

  dongjoon-hyun

  committed
  7a7b086

  Copy full SHA for 7a7b086

Commits on Jul 12, 2021

• [SPARK-36056][SQL] Combine readBatch and readIntegers in VectorizedRleValuesReader

  Show description for 5edbbd1
  

  sunchao

  authored and

  cloud-fan

  committed
  5edbbd1

  Copy full SHA for 5edbbd1

Commits on Jun 30, 2021

• [SPARK-34859][SQL] Handle column index when using vectorized Parquet reader

  Show description for a5c8866
  

  2 peopleauthored and

  dongjoon-hyun

  committed
  a5c8866

  Copy full SHA for a5c8866

Commits on Jun 23, 2021

• [SPARK-35846][SQL] Introduce ParquetReadState to track various states while reading a Parquet column chunk

  Show description for b8acbf6
  

  sunchao

  authored and

  dongjoon-hyun

  committed
  b8acbf6

  Copy full SHA for b8acbf6

Commits on Jun 16, 2021

• [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

  Show description for 506ef9a
  

  sunchao

  authored and

  dongjoon-hyun

  committed
  506ef9a

  Copy full SHA for 506ef9a

Commits on Jun 12, 2021

• [SPARK-35321][SQL] Don't register Hive permanent functions when creating Hive client

  Show description for 9c7250f
  

  sunchao

  authored and

  wangyum

  committed
  9c7250f

  Copy full SHA for 9c7250f

Commits on Jun 11, 2021

• [SPARK-35640][SQL] Refactor Parquet vectorized reader to remove duplicated code paths

  Show description for e9ccf4a
  

  sunchao

  authored and

  dbtsai

  committed
  e9ccf4a

  Copy full SHA for e9ccf4a

Pagination

Previous

Next