Support showing non UTF-8 characters #50
Description
Currently when a non UTF-8 character is displayed on stderr, I can't view the whole log.
Read error: stream did not contain valid UTF-8
But I think it should hide those characters instead of crashing. I saw those non UTF-8 characters while printing progress bars in multi-node runs.
Replacing attention layers: 69%|██████▉ | 33/48 [00:05<00:02, 5.75layer/s]
Replacing attention layers: 71%|███████ | 34/48 [00:06<00:02, 5.65layer/s]
Replacing attention layers: 73%|█████�[rank0]:[W1101 11:02:41.677904760 ProcessGroupNCCL.cpp:4561] [PG ID 0 PG GUID 0 Rank 0] using GPU 0 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. Specify device_ids in barrier() to force use of a particular device, or call init_process_group() with a device_id.
�█▎ | 35/48 [00:06<00:02, 5.67layer/s]
Replacing attention layers: 75%|███████▌ | 36/48 [00:06<00:02, 5.68layer/s]
Replacing attention layers: 77%|███████▋ | 37/48 [00:06<00:01, 5.82layer/s]