# Changelog

## [v0.1.0-alpha](https://github.com/AdRoll/baker/tree/v0.1.0-alpha) - 2021-03-19

### Added

- upload: add S3 uploader component [AdRoll#15](AdRoll#15)
- filter: add ClearFields filter [AdRoll#19](AdRoll#19)
- output: add Stats output [AdRoll#23](AdRoll#23)
- filter: add SetStringFromURL filter [AdRoll#28](AdRoll#28)
- output: add FileWriter output in replacement of Files output  [AdRoll#31](AdRoll#31)
- upload: s3: add `ExitOnError` configuration [AdRoll#27](AdRoll#27)
- uploads now return an error instead of panicking and baker deals with it [AdRoll#27](AdRoll#27)
- general: replace `${KEY}` in the TOML conf with the `$KEY` env var [AdRoll#24](AdRoll#24)
- input: add KCL input. [AdRoll#36](AdRoll#36)
- filter: add RegexMatch filter. [AdRoll#37](AdRoll#37)
- filter: add NotNull filter [AdRoll#43](AdRoll#43)
- filter: add Concatenate filter [AdRoll#28](AdRoll#33)
- Required configuration fields are now handled by Baker rather than by each component. [AdRoll#41](AdRoll#41)
- filter: add TimestampRange filter [AdRoll#46](AdRoll#46)
- filter: add ReplaceFields filter [AdRoll#49](AdRoll#49)
- filter: add Timestamp filter [AdRoll#54](AdRoll#54)
- Add Record.Copy method [AdRoll#53](AdRoll#53)
- Add pkg/splitwriter and pkg/buffercache reusable components [AdRoll#55](AdRoll#55)
- output: add SQLite output [AdRoll#56](AdRoll#56)
- README: document KCL input [AdRoll#59](AdRoll#59)
- Document how to specialize baker.LogLine [AdRoll#63](AdRoll#63)
- Add `baker.MainCLI` [AdRoll#73](AdRoll#73)
- Implement markdown rendering of component help/configuration [AdRoll#80](AdRoll#80)
- Add `[fields]` section in TOML in which use can define field indexes <-> names mapping [AdRoll#84](AdRoll#84)
- Add StringMatch filter which discards/keeps records based on the result of string comparisons  [AdRoll#102](AdRoll#102)
- Add PartialClone filter [AdRoll#107](AdRoll#107)
- Add `[validation]` section in TOML in which users can define record validation through regex [AdRoll#122](AdRoll#122)
- Add ExpandJSON filter [AdRoll#128](AdRoll#128)
- Add Hash filter with the support of md5 and sha256 algorithms [AdRoll#130](AdRoll#130)
- Add MetadataLastModified filter [AdRoll#133](AdRoll#133)
- Add Dedup filter [AdRoll#143](AdRoll#143)
- Add MetadataUrl filter [AdRoll#147](AdRoll#147)
- Add FormatTime filter [AdRoll#151](AdRoll#151)
- Add ExpandList filter [AdRoll#155](AdRoll#155)

### Changed

- Do not force GOGC=800, let inputs decide and user have final word [AdRoll#13](AdRoll#13)
- Move aws-specific utilities into a new `awsutils` package [AdRoll#14](AdRoll#14)
- Outputs' `Run()` returns an error [AdRoll#21](AdRoll#21)
- Fix 2 panics: ValidateRecord and errUnsuportedURLScheme [AdRoll#29](AdRoll#29)
- Remove datadog-specific code from [general] section. Instead add [metrics] which can be extended with baker.MetricsClient interfaces. [AdRoll#34](AdRoll#34)
- Remove duration parameter from baker.Main [AdRoll#62](AdRoll#62)
- TimestampRange filter accepts 'now' as range [AdRoll#106](AdRoll#106)
- Standardize the components' structs names [AdRoll#105](AdRoll#105)
- **Breaking** Change func FieldName to FieldNames (slice) as it allows to know the number of defined fields [AdRoll#110](AdRoll#110)
- ExpandList filter just forwards if the source field is empty [AdRoll#171](AdRoll#171)

### Removed

- output: remove the Files output in favor of the more generic FileWriter [AdRoll#31](AdRoll#31)

### Fixed

- Fix a bug in `logline.Copy` [AdRoll#64](AdRoll#64)
- Fix building on windows [AdRoll#115](AdRoll#115)
- Fix `list_test` with file URI to be compatible with windows paths [AdRoll#117](AdRoll#117)
- Fix `List` input, some `io.Reader`'s were left opened [AdRoll#118](AdRoll#118)
- Fix some bugs in `s3.s3UploadFile` [AdRoll#120](AdRoll#120)
- `SplitWriter` leaves some file descriptors open [AdRoll#119](AdRoll#119) and [AdRoll#121](AdRoll#121)
- `PrintHelper()` now supports map type as configuration parameter of a Baker component [AdRoll#138](AdRoll#138)
- `List` input did not consider drive letter on Windows paths [AdRoll#139](AdRoll#139)
- Do not insert newline after dots in generated help markdown [AdRoll#140](AdRoll#140)
- Fix data race in statsDumper [AdRoll#154](AdRoll#154)
- SetStringFromURL filter panics if MetadataURL was not set. [AdRoll#156](AdRoll#156)
- MetadataLastModified filter wrongly counts the number of processed records [AdRoll#157](AdRoll#157)
- `List` input did not correctly manage HTTP status codes [AdRoll#163](AdRoll#163)

### Maintenance

- input: Fixes `List` input not managing S3 "folders" [AdRoll#35](AdRoll#35)
- input: with [AdRoll#35](AdRoll#35) we introduced a regression that has been fixed with [AdRoll#39](AdRoll#39)
- upload: fixes a severe concurrency issue in the uploader [AdRoll#38](AdRoll#38)
 remove `output.RawChanSize`