Having two keys allows to initiate secure communication through public unprotected channels without the need to pass the private key to other side. As others asymmetric algorithms, RSA can be used in the following ways:
RSA has been searched for security vulnerabilities for over 30 years (see 1998 year review. The algorithm has a strong mathematical background and so far theoretically cannot be broken in acceptable time. However concrete implementations of RSA may be sensitive to side channel attacks, misinterpreting whom belongs the private key of the known public key and other potential security flaws that are not related to the algorithm itself.
The most sensitive targets are timing attacks (if it is possible to measure very precisely how much it took for CPU to do authentication) and power attacks (if it is possible to record the precise power consumption profile that reflects the sequence of commands, actually executed by CPU). This is especially dangerous if timing and power profiles are available with high accuracy, as seen by oscilloscope, directly attached to the authenticating device. Also, it is a big advantage for the attacker to have possibility to experiment with the exact copy of the hardware and software stack the victim uses. The attack is much more difficult over network due induced network latency; however the attack over very fast and small local network may be possible.
These vulnerabilities are closed by using "cryptographic blinding" countermeasures that make both timing and power data also dependent on "on the fly" generated random number, preventing an easy attack.
There were some publications claiming that the RSA has been cracked "on a server" using timing channel attacks, but in this work the server cooperated with intruder providing precise times on how long it took to run the algorithm. In other study, the authors did recovered a key by varying CPU voltage outside approved boundaries (that caused multiple power faults). Again, Bernstein (2001)  did recovered a key but that key was much shorter that the keys practically used in the industry. Some companies manufacture cryptographic extension cards that they think are more secure than software-based solutions and still protects sensitive keys even after intruder gets physical access to the server box.
The algorithm is especially valuable because it is not relying on secrets how does the algorithm works; the source code can be public and the authentication is still secure. Ultrastudio.org server is also managed using RSA keys for authentication.
d is a secret private key exponent.
The message m is encrypted using formula
where c is the encrypted message. The encrypted message is decrypted using formula
Encryption and decryption formulas show how to encode and decode a single integer. Bigger (or different) pieces of information are encoded by converting them into (potentially large) integers first. As RSA is not particularly fast, it is usually only to encrypts the key of some faster algorithm. After RSA decrypts the key, this supplementary algorithm uses it to decrypt the rest of the message.
RSA algorithm is fundamentally based on the Euler theorem:
where a and n are positive integers and a is a co-prime to n.
To break the algorithm from the mathematical side, one needs to solve the factoring problem (find the two prime numbers that, when multiplied, produce the given result). When the picked numbers are large enough, the problem cannot be easily solved by brute force and at least currently it also does not have easier analytic solution.