Bitcoins are not stored on your PC.
The ledger (blockchain) keeps track of every transaction and therefore keeps tracks of the UTXO (unspent outputs = 'available coins').
You could say the coins are 'stored on the blockchain' (That might be true in a logical sense, but is still not completely true on a technical level).

Now to awnser your question:
You need a private key, corresponding to the address 'which has funds on'.
Usually private keys are managed within a wallet (wallet = piece of software which manages private/public keys, transactions, ...).
You can easily import the same private key into several different wallets (to be able to access them from different pc's).
Depending on the wallet you should also be able to copy the wallet file (this enables you to additionally also recieve funds to 'both pc's').
Some wallet may have sync problem when working with them in that way. This won't let you lose any funds, but might lead to some displaying issues.

Long answer short: Yes, you can.