It's easy to turn an app off on the tablet. Touch the apps button and swipe an app or apps off the screen.
But with the phone, turning an app off isn't so straightforward.
On the phone, swiping an app off the running apps screen doesn't work, or you didn't try? It works for me (CyanogenMod 12.1). You should be able to open it by long press of home button.