This commit contains changes exposed by running it any new environment (eg need to create var dir if it doesn't exist) and then specific changes needed when testing expanded from macos only to Linux and FreeBSD

update test tools to use /usr/bin/env to set interpretter

etc/README.txt - explains commands and environment expectations
bin/runenv - script to set environment variables to the most common value before executing
bin/check-env.pl - script to evaluate environment for you

bin/run-all.pl - mkdir var if it's not already present
bin/run-tests.pl - add handling for missing or incorrect PAGER variable
- notify when invalid option entered

Update test cases to be more consistent
- munge open2 failure output, debian and macos have different errors
- munge tls protocols which may not be consistent between test platforms
- update reference output from all tests that included --support now that we're requiring all modules be installed

Changes specifically for Windows:
- General user experience improvements - add runenv.bat, Windows exceptions in check-env.pl, Windows specific notes in README.txt, etc
- rework entire capture system, didn't work at all on Windows
- turn off stdout buffering so the run-tests.pl prompts will show in cmd.exe
- remove reliance on unix tools (cat, diff, cp, diff)
- tons of work to handle paths in a platform agnostic way
- replace %DATE% in some tests, that's a valid var in Windows (oops - need to address this more broadly in future)
- require that swaks end in .pl on Windows when testing. If test swaks doesn't have .pl extension, runenv.bat will make a copy of it at var/swaks.pl and use that for testing
- Don't chmod on Windows
- Add a swaks.pl wrapper to version control that just calls swaks.  Not really testing specific, but makes Windows development a little easier
- There's no current obvious solution for expect on Windows.  For now, just skip and test with an INTERACT verb in it on Windows

Changes that can be used on any platform but the need for them was exposed by Windows work:
- Allow individual test lines to be conditional based on current platform (IFOS)
- add %QUOTE_DOUBLE% and %QUOTE_SINGLE% tokens which are replaced in test case lines after shellwords parsing and before execution.  There were some complex quoting issues on Windows I couldn't otherwise resolve
- run-all.pl will print a warning if an entire test-suite dies
- run-tests.pl will always turn the test swaks into an absolute path.  Windows did this anyway - forcing it everywhere made some things easier.
- Allow custom tokens to be defined by and then used by test cases.  The feature is not platform specific but it was needed to allow us to create a per-platform name for a custom swaks (so that it can end in .pl on Windows) without having to make a ton of lines platform specific.
- Munge automatically-generated usernames in test results.
- run-all.pl - print a summary of test results at the end of a run.

-  Allow LOGNAME to be used on Windows.  It's not a concept native to Windows, but there's no reason not to allow the default sender to be overridden by an environment variable
-  Fix date generation on Windows.  Windows supports strftime, but not the %z format token we were using.  Dates will be GMT on Windows and any other platform that doesn't support %z.
-  Fix how the app name is found - was broken both by the change in file system delimiters and the .pl extension
-  use binmode on STDIN and STDOUT to prevent perl helping with line ending translations
-  change the #! line to use /usr/bin/env for portability (this might have been for FreeBSD actually)
-  Fix setting environment variables on windows
--- since env vars are caseless on Windows, force all options to lower case (Note that this breaks -S.  Document and try to avoid case collisions in options in the future).
--- Since Windows doesn't allow empty environment variables, allow variable values of '<>' to mean "set but empty"
--- Explicitly revoke support for header names embedded in the --header option when set via environment variable on Windows (because Windows vars are caseless)

* portability:
  Add alternate token form ..TOKEN..
  move help generation to Pod::Usage from manual call to perldoc
  Improve swaks support for Windows
  testing - extensive rework to support Windows cmd.exe
  testing - add --winnow option to run-all.pl
  testing - make automated tests easier to run in new environments

1) Now that we have ..DATE.. available, change the ones I changed to MESSAGE_ID back to ..DATE..
2) Because I wasn't running them on windows, I never changed the INTERACT tests.  Change them to ..DATE.. also

1) add sle.pl and slediff.pl utilities for a way to evaluate line ending diffs that's easier than looking at xxd output
2) Add new diff options to run-tests.pl.
  - (w) - show the diff with \r and \n characters made visible (useful for understanding why a regular diff shows changes but looks unchanged
  - (o) - before diffing files, normalize all line endings to "\n".  This is useful for evaluating changes when there is a misture of line ending and non-line-ending changes

Just use the already-existing ability to capture output from and write input to an external script.  Allows us to enable interactive tests on windows

Pros: totally cross-platform, including addressing line ending issues we were experiencing on Windows
Cons: we're not actually responding directly to prompts.  This is slightly more fragile, but I think it's fine for a set of test tools.

* windows-expect5:
  Remove reliance on external tool for interactive tests
  Improvements to testing tools related to line endings
  Fix tests related to %DATE%

… release

closes #13

closes #24

* 24-truncate-output-files:
  output-file family of options should truncate file if it exists

…r and --local-interface options

* codify-port-priority:
  --port and --local-port options should override ports set via --server and --local-interface options

set up for "signed" and unsigned certs to allow testing of crt verification

Exim and Postfix

Postfix:
_NOTES, _START.sh, _STOP.sh are all from me, everything else is from a 'make install' from postfix source.  aliases, main.cf, and master.cf have been modified (see _NOTES/install-notes.txt)

misc
- remove outdated testing notes
- get rid of fake Authen::NTLM
- adjust tests which show auth to include Authen::NTLM options

runenv
- make subl the default editor
- only include the fake Authen::NTLM if the real one isn't present

runenv.bat
- specify the smtp tester

run-all.pl
- allow to run a selection of test suites

trans-to-test.pl
- new tool to translate actual swaks transaction snippets into something usable by the test server

run-tests.pl
- add support for transaction testing (add TEST_SERVER expansion and clean up some munging)
- add munge_tls_cipher
- add munge for show-time-lapse
- add FORK verb to allow process to run in background during test
- add %CERTDIR% token

smtp-server.pl
- allow multiple test scripts to be run, and make the script-runner honor --silent
- convert smtp-server.pl to use Getopt::Long to allow future expansion
- handle both the hash form and the file form in the script runner (needed to allow the plain includes from scripts)
- support per-file tokens
- make cert/key configurable
- update so that files with no paths are assumed to be relative to the smtp-server scripts directory
- used signed keys under testing/certs by default

* transaction-3:
  Create transaction testing
  Improve testing tools and add support for transaction testing
  Config files for building MTAs for test reference
  Consolidate test certs under testing/certs

… attribute in the error

--drop-after-send connect -- never worked, now drops after connect but before reading banner
--drop-after-send tls (with --tlsc) never sent quit, now mimics --drop-after since we don't have access to the TLS negotiation
--drop-after-send helo will read the helo response if there's only one helo in the transaction.  Document this behavior and suggest first-helo instead
--drop-after-send tls (without --tls) never sent quit, now mimics --drop-after
--drop-after-send auth (without --auth) never sent quit, now mimics --drop-after

…XCLIENT is not negotiated.

another openssl upgrade on my laptop, another form of this error

Improve TLS transaction debugging around client and peer certificates

…-cert file

Also add a partial chain file to certs which is needed to test the --tle-get-peer-chain option

This is a bug fix for d1b104b, that change was too broad and munged "SSL routines::no ciphers available" to "SSL routines:: ciphers available".

add --tls-get-peer-chain

The goal of this action is to produce a copy/pastable version of the comand that was run for human debugging

bugfix: it did not properly preserve quoted arguments.  It turned "--pipe 'smtp-server.pl --arg 1 --foo 2'" into "--pipe smtp-server.pl --arg 1 --foo 2".  It now maintains the quoting
improvement: now adds backslashes and newlines after every option/arg pair to make it way easier to read

no logic changes, just breaking the starttls method into logical groupings to make it easier to work on

Previously I saved copies of each cert I saw in the verify callback function.  This had all sorts of problems, including sometimes causing duplicates to appear, sometimes causing certs to appear that weren't presented (I think these were certs from the local CA, though this was not reliable enough to be useful), and getting different verification callbacks between macos/debian and freebsd, meaning testing wasn't working well on freebsd.

The solution is to skip getting the certs from verify and just get them directly from get_peer_cert_chain.  This is way, way more reliable and reproducible

  - remove notBefore
  - move notAfter, serverAltName, and commonName to one line
  - wrap all in [], get rid of spaces cushioning subjectAltName
  - change DN= to subject=, wrap it in [] instead of double quotes
  - use peer/local[0] regardless of whether there's more than one cert

Rework tls internals

…dle multiple actions

Clean up

Allow to see unmunged commands that are executed for the test without needing the test to fail

…onnection right after

* 20240102 Rework tls verification. On cert verification failure, print openssl's error message.  Get rid of verify callback.
* 20240102 Clarify --tls-verify-ca docs, this verifies both signing and date expiration (notBefore/notAfter)

TLS improvements (#66, #76)

remove previously-deprecated option -g

…e different but valid output

some of these are needed because --help renders the pod differently on freebsd
some are needed because the different version of openssl has different error messages when tls verification fails

I suspect these are version specific, not OS specific, but for the time being this is a convenient way to mask the differences out

…stsuite errors on freebsd

Fix up freebsd testing

Release/v20240103.0