By default, Prison Architect will not delete leftover mods once you deactivated them in the game. This may lead to strange problems now and then, so in order to get rid of old stuff you can follow this guide.
Removing Mods Manually (Step-by-Step)
Open Save Folder
First, open the folder where the local user files are stored.
You can do this by going to Options, then click Open Save Folder and exit the game.
A file manager like Windows Explorer will open, showing you the contents of this map.
The Mods Map
As you can see there are several maps presented in the window:
The 'mods' map contains all mods to which you've subscribed in the past.
It will contain unsubscribed, deactivated old mods until you remove them manually.
When you go into this folder, you will see all kinds of subfolders with numbers.
The numbers you see correspond to the number in the Steam Workshop URL from that mod.
Open such a folder and you see a manifest.txt which holds a description.
If you want to remove a mod, simply delete this numbered folder.
You can also create a new folder like 'mods - backup' and move mods to that location.
Backup folders like this are not loaded by the game, it doesn't know anything about them and won't mess up your game. Call it any way you like.
The screenshot shows you 'mods - backup' and 'mods - various'.
Both folders contain stuff which I conveniently store there.
You can also just completely remove or rename the 'mods' folder itself.
The game will recreate this map when it doesn't exist.
The main folder also holds the preferences.txt which you can edit when the game is not running.
Inside you can see a line called Mods which hold the mods you've activated in the game.
These are in the load order of the mods list.
Mods on top overrule mods below them, so in this example 3 mods overrule the combipack.
Remove the characters, but not the spaces, change it to (empty) to instantly deactivate all mods.
Save the file and you're done!
There is also a debug.txt located in this data folder.
Open it to see what happened while starting the game.
Sometimes it can hold valuable information for you to pinpoint certain issues!
If your game crashes, then you can find crashdump files where the game was installed.
This is usually C:\Program Files (x86)\Steam\steamapps\common\Prison Architect
Instead of moaning that the game won't start, you can upload these dump files together with the debug.txt so the devs can have a look at the issue and most likely even fix it for a next game update.