I'm not sure how to respond, or what might help.
Just a few random thoughts--I'm brainstorming here so if none of these ideas sound doable maybe something else will occur.
First, if he's abusing you in the van, I would take this up with the management at the company providing the transport. I of course don't know the details, but it sounds like you use some form of para-transit to get to the program. Where I live all those vans are owned by a company that the state contracts with for the service. I would think that it's the company's obligation to provide a safe environment for its passengers. The driver should have stepped in and done more than just turn up the music. He should have stated clearly that abusive language isn't permitted. If nothing else, an argument like that is probably very distracting, which could become a safety issue.
So I would find out what sort of complaint process is in place with the transit people, and pursue that, being careful not to criticize the driver but rather to focus on the abusive passenger. HE should be contacted by the company and told his behavior will not be tolerated. It would help if you could get at least one other passenger to sign on to the complaint, but if not it would still be worth doing. If nothing else it establishes a written record of his abusive behavior which might be useful in the future.
I'd also register a complaint with whoever is in overall charge of your program. If you don't feel safe in the program, and if others have already left because of the abuse, the supervisors at the program should be informed. I would encourage your friend who dropped out of the program to contact them to say why. I would think the program, like the van company, has an obligation to provide for a safe environment.
Whatever you write should be polite but to the point. Keep copies for yourself, and keep copies of whatever response you get.
As for the last part--if he threatens you I'd go beyond talking to the landlord. I'd call the police (again, if nothing else then just to establish a "paper trail" as the basis for a restraining order).
Finally, I'd find out if your state and locality (city, town, village) has either a disability rights or human rights commission. I'm not sure but you may have some options there. If you feel you're being abused or discriminated against because you're trans, this might be something they would want to look into. Again, if nothing else it puts all this on the record, which could come in useful if the nasty and threatening behavior continues or even escalates.
Those are my thoughts, and I hope they're helpful.
Let me say again how sorry I am that you have to go through this shit.
Best wishes.