2024 Update:
Firefox now have support for vertical tabs built in, and I recommend using it instead of Sidebery:
NB: It is worth noting that these will not auto-hide, you have to press the “Expand sidebar”-button at the top left manually.
- Open
about:config- “Accept the Risk and Continue”
- Enable
siebar.verticalTabs
Firefox can look old-school, but with some extra add-ons and styling it can get a fresh look with auto-hiding vertical-tabs:

The following steps will:
NB: At the time of writing, the following modifications require beta-version 5.0.0b31, which must be downloaded and installed from the GitHub releases page.
Open the URL about:support in Firefox, locate Profile Directory and press the Open Directory-button.

If necessary create a folder named chrome, and then inside it a file called userChrome.css.
Modify the chrome/userChrome.css-file with the attached userChrome.css.
Open the URL about:config in Firefox and enable toolkit.legacyUserProfileCustomizations.stylesheets.
Open the Sidebery settings by pressing the cogwheel-icon in the upper right corner of Sidebery and navigate to the Styles editor.
Paste the content of the SideberyStyles.css into the editor (on the right side of the page).
Right click the toolbar and press Customize Toolbar…. At the bottom-left side of the page uncheck Title Bar and press the Done-button.
Restart Firefox to reload the userChrome.css-file.
At the time of writing pinned tabs are rendered with weird icon-spacing in Sidebery. To counteract this behaviour, you can enable showing titles for pinned tabs, making them render vertically instead of horizontal (personally I find it preferable anyway).
Open Sidebery settings and under Settings > Tabs > Pinned tabs enable Show titles of pinned tabs.