It's easy to turn an app off on the tablet. Touch the apps button and swipe an app or apps off the screen.
But with the phone, turning an app off isn't so straightforward.
On the phone, swiping an app off the running apps screen doesn't work, or you didn't try? It works for me (CyanogenMod 12.1). You should be able to open it by long press of home button.
Thanks for telling me, now I see how to swipe apps away.
On my Oneplus One phone running cm 12.0, long-pressing the menu key ("hamburger" icon) shows running apps.
They can be swept from there.