What's Changed

• fix hatch build by explicitly specifying directories to include and excluding my/ symlink by @karlicoss in #439
• general: sort out F401 (unused variable) warnings in ruff everywhere by @karlicoss in #438

Full Changelog: v0.6.20250622...v0.6.20250623

What's Changed

General

Some notable upgrades -- shouldn't break anything for existing users, but worth highlighting:

• general: switch from flat layout to src layout by @karlicoss in #414
  If something isn't working (even though I made sure everything is compatible), best to reinstall your HPI
• core.common: use kompress directly instead of attempting to import previously vendorised version by @karlicoss in #422
• kompress: get rid of vendorised version by @karlicoss in #417
  If something isn't working, make sure to run pip install kompress
• core: migrate from deprecated appdirs to platformdirs by @karlicoss in #418
  If something isn't working, run pip install platformdirs

Other minor changes:

• ci: bring pytest config/pyproject/tox up to date/switch to hatch for building package by @karlicoss in #437
• ci: switch to pyproject.toml and update other CI files by @karlicoss in #416
• general: use Exception.add_note instead of hacky my.core.error.echain by @karlicoss in #421
• my.core.experimental: some fixes from import_original_module by @karlicoss in #433
• general: remove unnecessary init.py in tests/ subpackages by @karlicoss in #434

Core

• core: add my.core.json.json_loads, use orjson if available, otherwise fall back to stdlib json by @karlicoss in #425
• core: migrate code to benefit from 3.9 stuff by @karlicoss in #401
• general: migrate modules to use 3.9 features by @karlicoss in #402
• minor updates: remove assert_subpackage and clean up some ruff by @karlicoss in #420

Modules

• my.whatsapp.android: adapt to new db format by @karlicoss in #403
• fix typo in rexport DAL by @srajangarg in #405
• smscalls: add phone number to model by @purarue in #410
• smscalls: allow large XML files as input by @purarue in #411
• core.sqlite: add helper SqliteTool to get table schemas by @karlicoss in #412
• twitter.android: fix get_own_user_id for latest exports by @karlicoss in #413
• instagram.gdpr: some cleanup and properly extract thread_v2_id by @karlicoss in #423
• instagram.android: improve thread id extraction and add more comments by @karlicoss in #424
• my.lastfm: some cleanup by @karlicoss in #426
• new module: podcastaddict android app by @karlicoss in #429
• fbmessenger.android: fix missing senders, fallback on empty name by @karlicoss in #430
• my.bluemaestro: support new bmLogger app by @karlicoss in #431
• my.bumble.android: some cleanup + switch to Protocol-based config by @karlicoss in #432
• twitter.android: handle empty databases by @karlicoss in #435

Misc

• chore: update urls by @purarue in #404
• doc: spelling fixes by @purarue in #408
• doc: some performance notes for query_range by @purarue in #409
• src/ doc links by @PatMyron in #415
• general: update mypy rules and fix by @karlicoss in #419
• fix ruff error by @karlicoss in #428
• ci: sync ruff rules with base config and fix warnings by @karlicoss in #436
• workaround to make pip install hpi[testing] work while dependency group support is missing in pip by @karlicoss in #427

New Contributors

• @srajangarg made their first contribution in #405
• @PatMyron made their first contribution in #415

Full Changelog: v0.5.20241019...v0.6.20250622

What's Changed

• general: python3.9 reached EOL, switch min version by @karlicoss in #400

Full Changelog: v0.5.20240924...v0.5.20241019

What's Changed

• core.time: hotfix for default force_abbreviations attribute by @karlicoss in #399

Full Changelog: v0.5.20240923...v0.5.20240924

What's Changed

core

• my.core: fix list constructor in always_support_sequence and add some tests by @karlicoss in #395
• my.core.structure: add support for .tar.gz archives by @karlicoss in #390

modules

• my.google.takeout.parser: speedup event merging on newer google_takeout_parser versions by @karlicoss in #389
• my.github.gdpr/my.zulip.organization: use kompress support for tar.gz if it's available by @karlicoss in #392
• my.fbmessenger.android: exclude unsent messages to avoid duplication by @karlicoss in #393
• migrate bluemaestro and pdfs configs to use Protocol and @Property by @karlicoss in #384
• migrate some modules to 'lazy config' pattern and clean up tests by @karlicoss in #386
• small fixes for my.twitter.android, dedupicate tweets in my.twitter.archive by @karlicoss in #394
• my.youtube.takeout: deduplicate watched videos and sort out a few minor errors by @karlicoss in #397

misc

• update ruff config, enable/suppress some checks by @karlicoss in #387
• ci: update mypy config and make ruff config more consistent with other projects by @karlicoss in #388
• tox: some prep for potentially using uv on CI instead of pip by @karlicoss in #396
• tox: try using uv for CI, should result in speedup by @karlicoss in #398

Full Changelog: v0.5.20240824...v0.5.20240923

What's Changed

Mostly big cleanup & deprecations for core.common module #380 #382

New core dependency: typing-extensions

Lots of cleanup/deprecations in core. This mostly shouldn't break any runtime code, but mypy might complain -- so best to migrate the imports appropriately

• core.common: deprecate outdated LazyLogger alias

• core.common: move Json, datetime_aware, datetime_naive, is_namedtuple, asdict to my.core.types

• my.core: deprecate Path/dataclass imports from my.core during type checking

  runtime still works for backwards compatibility

• core.common: move assert_subpackage to my.core.internal

• core.common: move mime-related stuff to my.core.mime

  no backward compat, unlikely it was used by anyone else

• core.common: move stats-related stuff to my.core.stats and add more thorough tests/docs

  deprecate core.common.stat and core.common.Stats with backwards compatibility

• core: cleanup my.core.common.unique_everseen

  • move to my.core.utils.itertools
  • more robust check for hashable types -- now checks in runtime (since the one based on types purely isn't necessarily sound)
  • add more testing

• my.core.common: move warn_if_empty to my.core.utils.itertools, cleanup and add more tests

• core.common: move away import related stuff to my.core.utils.imports

  moving without backward compatibility, since it's extremely unlikely they are used for any external modules

  in fact, unclear if these methods still have much value at all, but keeping for now just in case

• core.common: move listify to core.utils.itertools, use better typing annotations for it

  also some minor refactoring of my.rss

• core: cleanup itertool style helpers

  • deprecate group_by_key, should use itertool.bucket instead
  • move make_dict and ensure_unique to my.core.utils.itertools

• core: cleanup deprecations, exclude from type checking and show runtime warnings

  among affected things:

  • core.common.assert_never
  • core.common.cproperty
  • core.common.isoparse
  • core.common.mcachew
  • core.common.the
  • core.common.tzdatetime
  • core.compat.sqlite_backup

• core: cleanup deprecations, exclude from type checking and show runtime warnings

  among affected things:

  • core.common.assert_never
  • core.common.cproperty
  • core.common.isoparse
  • core.common.mcachew
  • core.common.the
  • core.common.tzdatetime
  • core.compat.sqlite_backup

Full Changelog: v0.4.20240810...v0.5.20240824

What's Changed

• fix for new instagram gdpr format by @karlicoss in #371
• fix legacy google takeout by @karlicoss in #372
• fbmessenger.android: fix minor issue with processing thread participants by @karlicoss in #373
• fix my.twitter.android by @karlicoss in #374
• tests: use updated conftest from pymplate, this allows to run individual test modules properly by @karlicoss in #375
• improve testing consistency, simplify tox by @karlicoss in #376
• replace deprecated utcnow and utcfromtimestamp by @karlicoss in #378

Full Changelog: v0.4.20240506...v0.4.20240810

What's Changed

• monzo.monzoexport: initial module by @karlicoss in #346
• time.tz.via_location: more consistent behaviour wrt caching by @karlicoss in #351
• my.codeforces/my.topcoder: move to the top level & cleanup by @karlicoss in #352
• my.body.sleep: massive speedup for average temperature calculation by @karlicoss in #353
• gpslogger: catch broken xml file error by @seanbreckenridge in #356
• initial module for official twitter app for android by @karlicoss in #359
• my.youtube.takeout: fix exception handling by @karlicoss in #361
• fbmessenger.android: expose contacts by @karlicoss in #362
• whatsapp.android: expose all entities extracted from the db by @karlicoss in #363
• google takeout: support multiple locales, resolves #330 by @seanbreckenridge in #364
• my.google.maps: intitial module for extracting saved places data from Android app by @karlicoss in #365
• my.twitter.android: better detection of own user id by @karlicoss in #368
• my.tinder.android: better error handing and fix case with empty db by @karlicoss in #369
• smscalls: parse mms from smscalls export by @seanbreckenridge in #370

Full Changelog: v0.4.20231102...v0.4.20240506

not ideal that the date in the version is the future date.. but oh well

What's Changed

• ci: fix wrong release command by @karlicoss in #344

Full Changelog: v0.4.20231101...v0.4.20231102

Big update, even though things should be backwards compatible, it's recommended to install the package again if you use editable git checkout (pip3 install --user -e /path/to/hpi_repo)

Also worth installing optional dependencies (pip3 install --user -e /path/to/hpi_repo[optional])

What's Changed

Core:

• core: implement more methods for ZipPath and better support for get_files by @karlicoss in #285
• query: add --warn-exceptions, dateparser, docs by @seanbreckenridge in #290
• my.core.pandas: rely on typing annotations from types-pandas by @karlicoss in #292
• core: experimental import of my._init_hook to configure logging/warnings/env variables by @karlicoss in #298
• core/logging: overhaul and many improvements -- mainly to deprecate abandoned logzero by @karlicoss in #299
• minor module updates + logging/warnings improvements by @karlicoss in #300
• general: move reddit tests into my/tests + tweak my.core.cfg to be more reliable by @karlicoss in #295
• core/cli: allow user to bypass PEP 668 by @seanbreckenridge in #302
• core/cli: update vendored completion files by @seanbreckenridge in #304
• core/logging: fix issue with logger setup called multiple times when called with different levels by @karlicoss in #310
• my.core.logging: compatibility with HPI_LOGS by @seanbreckenridge in #307
• core/stats: exclude contextmanagers from guess_stats by @karlicoss in #334
• move hpi specific compat stuff to core.hpi_compate + minor updates for hypothesis and kobo by @karlicoss in #335
• core/config: implement a warning if config is imported from the dir other than MY_CONFIG by @karlicoss in #336
• general: improve logging during file processing in various modules by @karlicoss in #339
• general: enhancle logging for various modules by @karlicoss in #340
• general: deprecate some old methods by hiding behind TYPE_CHECKING by @karlicoss in #342
• core: add helper for more_iterable to check that all types involved are hashable by @karlicoss in #343
• core/kompress: move vendorized kompress to _deprecated, use kompress library directly by @karlicoss in #319
• more consistent handling of zip archives in get_files, less kompress boilerplate in modules by @karlicoss in #321
• experimental: add a hacky helper to import "original/shadowed" modules from within overlays by @karlicoss in #322
• core/stats: less duplication in output, report stats of first item, report input stats by @karlicoss in #324
• core.logging: ignore CollapseLogsHandler if we're not attached to a terminal by @karlicoss in #329
• dcotor: make compileall check a bit more defensive by @karlicoss in #332

modules:

• my.whatsapp.android: initial module by @karlicoss in #284
• smscalls: make some fields optional by @seanbreckenridge in #287
• cli: add option to output locations to gpx files by @seanbreckenridge in #286
• smscalls: make checking for keys stricter by @seanbreckenridge in #289
• browser: fix duplicate logs when fetching loglevel by @seanbreckenridge in #309
• new module: Harmonic app for Hackernews by @karlicoss in #313
• location: make accuracy default config floats by @seanbreckenridge in #314
• reddit.rexport: big cleanup + use new types from rexport.dal by @karlicoss in #320
• instagram: better normalising/error handling by @karlicoss in #325
• updates for whatsapp.android and tinder.android by @karlicoss in #326
• twitter.archive: deduplicate results via json.dumps by @karlicoss in #327
• reddit.rexport: experiment with using optional cpu pool (can potentially be used by other HPI modules) by @karlicoss in #328
• small zulip and dogsheep updates by @karlicoss in #331
• my.emfit: cleanup and pass cpu pool by @karlicoss in #333
• my.stackexchange.stexport: small cleanup & stat improvements by @karlicoss in #337
• fbmessenger.android: support processing msys database by @karlicoss in #341

Full Changelog: v0.3.20230327...v0.4.20231101