See https://github.com/timothylcooke/AvalonDock-Viewbox for repro.
When a LayoutAnchorable is shown within a LayoutAutoHideWindowControl, and the DockingManager is a descendent of a Viewbox, the LayoutAutoHideWindowControl is not a descendent of the Viewbox, so its LayoutAnchorable is not properly scaled, and dragging to resize it causes strange behavior.

FloatingWindow is also not zoomed, but the behavior is not nearly as strange because the FloatingWindow does not appear to be a descendent of the DockingManager in the same way that a LayoutAutoHideWindowControl does.