Hi, sorry to bother you
I’m exploring how to implement Hierarchical Reinforcement Learning (HRL) within the Tianshou framework for a project involving multi-level decision-making tasks. Probably the exact same question as #805
I’d appreciate guidance on extending it for hierarchical setups.
Specifically for :

1. How can I structure high-level policies and low-level policies within Tianshou’s Policy class?
2. What is the recommended way to handle high-level and low-level policies ?
3. How should hierarchical interactions be managed in the Collector and Trainer classes?
4. Is there support for multi-phase sampling (e.g., high-level decisions feeding into low-level transitions)?
5. Are there limitations in Tianshou’s current design that I should be aware of for HRL?

Thank you for your time, and I would greatly appreciate your response.