"When it's running" also means when it runs in the background, and apps can wake themselves up any time they want when you allow them to send you notifications.
Running location triggers an icon in the UI which puts it in a list of apps that have run location recently. An app abusing this in the background will eventually cause the operating system to ask you “hey, are you okay with this?” alongside a map of your current location as your phone sees it.
