If it is being used in Direct and Indirect Speech , it would be >> He told us to go . while in if its in simple conversion from active to passive voice then it would be like >>> Let ourselves be gone .

Please correct if someone in the forum finds it wrong

Regards