The parameter "max_grad_norm" is optional in the PPOPolicy parameter list, but in the training phase, PPOPolicy will require that the value of max_grad_norm cannot be None (but the default value of max_grad_norm is None).