Some exchanges have order-types that allow you to do that. Kraken, for example, has a "Stop" order that does exactly what you want. There might be other exchanges with the same features, but since I don't know any, I'll let someone else name them