📚 Math in Society
⇩ Download ▾

16.5 Public Key Cryptography

Suppose that you are connecting to your bank’s website. It is possible that someone could intercept any communication between you and your bank, so you’ll want to encrypt the communication. The problem is that all the encryption methods we’ve discussed require than both parties have already agreed on a shared secret encryption key. How can you and your bank agree on a key if you haven’t already?

This becomes the goal of public key cryptography – to provide a way for two parties to agree on a key without a snooping third party being able to determine the key. The method relies on a one-way function; something that is easy to do one way, but hard to reverse. We will explore the Diffie-Hellman-Merkle key exchange method.

As an example, let’s consider mixing paint. It’s easy to mix paint to make a new color, but much harder to separate a mixed paint into the two original colors used.[1][2]

Diagram of the paint mixing analogy for the Diffie-Hellman-Merkle key exchange, drawn as two columns headed Alice and Bob. Both begin with the same yellow common paint; each then adds a secret colour, orange-red for Alice and teal for Bob, giving Alice an orange mixture and Bob a blue one. Crossed arrows labelled public transport swap the two mixtures, annotated with the note that separating a mixture is expensive. Each person stirs their own secret colour into the mixture they received, and both columns end with the same dark olive tin labelled common secret.Using this analogy, Alice and Bob publically agree on a common starter color. Each then mixes in some of their own secret color. They then exchange their mixed colors.

Since separating colors is hard, even if a snooper were to obtain these mixed colors, it would be hard to obtain the original secret colors.

Once they have exchanged their mixed colors, Alice and Bob both add their secret color to the mix they obtained from the other person. In doing so, both Alice and Bob now have the same common secret color, since it contains a mix of the original common color, Alice’s secret color, and Bob’s secret color.

They now have a common secret color they can use as their encryption key, even though neither Alice nor Bob knows the other’s secret color.

Likewise, there is no way for a snooper to obtain the common secret color without separating one of the mixed colors.

To get this process to work for computer communication, we need to have the process result in a share common number to act as the common secret encryption key. For this, we need a numerical one-way function.

Modular arithmetic

If you think back to doing division with whole numbers, you may remember finding the whole number result and the remainder after division.

Modulus[3]

The modulus is another name for the remainder after division.

For example, 17 mod 5 = 2, since if we divide 17 by 5, we get 3 with remainder 2.

Modular arithmetic is sometimes called clock arithmetic, since analog clocks wrap around times past 12, meaning they work on a modulus of 12. If the hour hand of a clock currently points to 8, then in 5 hours it will point to 1. While 8+5=13, the clock wraps around after 12, so all times can be thought of as modulus 12. Mathematically, 13 mod 12 = 1.

Recall that when we divide 17 by 5, we could represent the result as 3 remainder 2, as the mixed number 325, or as the decimal 3.4. Notice that the modulus, 2, is the same as the numerator of the fractional part of the mixed number, and that the decimal part 0.4 is equivalent to the fraction. We can use these conversions to calculate the modulus of not-too-huge numbers on a standard calculator.

The one-way function

When you use a prime number p as a modulus, you can find a special number called a generator, g, so that gn mod p will result in all the values from 1 to p1

n 3 n 3 n m o d 7
133
292
3276
4814
52435
67291

In the table to the top, notice that when we give values of n from 1 to 6, we get out all values from 1 to 6. This means 3 is a generator when 7 is the modulus.

This gives us our one-way function. While it is easy to compute the value of gn mod p when we know n, it is difficult to find the exponent n to obtain a specific value.

For example, suppose we use p=23 and g=5. If I pick n to be 6, I can fairly easily calculate 56 mod 23=15625 mod 23=8

If someone else were to tell you 5nmod23=7, it is much harder to find n. In this particular case, we’d have to try 22 different values for n until we found one that worked – there is no known easier way to find n other than brute-force guessing.

While trying 22 values would not take too long, when used in practice much larger values for p are used, typically with well over 500 digits. Trying all possibilities would be essentially impossible.

The key exchange

Before we can begin the key exchange process, we need a couple more important facts about modular arithmetic.

You may remember a basic exponent rule from algebra:

( a b ) c = a b c = a c b = ( a c ) b

For example:

64 2 = ( 4 3 ) 2 = 4 6 = ( 4 2 ) 3 = 16 3

We can combine the modular exponentiation rule with the algebra exponent rule to define the modular exponent power rule.

This provides us the basis for our key exchange. While it will be easier to understand in the following example, here’s the process:

  1. Alice and Bob agree publically on values a prime p and generator g.
  2. Alice picks some secret number a, while Bob picks some secret number b.
  3. Alice computes A=gamodp and sends it to Bob.
  4. Bob computes B=gb mod p and sends it to Alice.
  5. Alice computes Bamodp, which is (gbmodp)amodp.
  6. Bob computes Abmodp, which is (gamodp)bmodp.

The modular exponent power rule tells us (gamodp)bmodp=(gbmodp)amodp, so Alice and Bob will arrive at the same shared value to use as a key, even though neither knows the other's secret number, and no eavesdropper can determine this value knowing only g,p,A, and B

RSA

There are several other public-key methods used, including RSA, which is very commonly used. RSA involves distributing a public encryption key, which anyone can use to encrypt messages to you, but which can only be decrypted using a separate private key. You can think of this as sending an open padlock to someone – they can lock up information, but no one can unlock it without the key you kept secret.

RSA’s security relies on the difficulty of factoring large numbers. For example, it’s easy to calculate that 53 times 59 is 3127, but given the number 12,317 that is a product of two primes, it’s much harder to find the numbers that multiply to give that number. It’s exponentially harder when the primes each have 100 or more digits. Suppose we find two primes p and q and multiply them to get n=pq. This number will be very hard to factor. If we also know p and q, there are shortcuts to find two numbers e and d so that medmodn=mmodn for all numbers m. Without knowing the factorization of n, finding these values is very hard.

To use RSA, we generate two primes p and q and multiply them to get n=pq. since we know the factorization, we can easily find e and d so med=mmodn. Now, we lock away p
q, and d. We then send the values e and n out publically. To encrypt a message m, the sender computes S=memodn. As we saw earlier, the modulus is a one-way function which makes the original message very hard to recover from S. However, we have our private key d we can use to decrypt the message. When we receive the secret message S, we compute Sdmodn=(me)dmodn=medmodn=mmodn, recovering the original message[4].

This method differs from Diffie-Hellman-Merkle because no exchange process is needed; Bob could send Alice an encrypted message using Bob’s public key without having to communicate with Alice beforehand to determine a shared secret key. This is especially handy for applications like encrypting email, where both parties might not be online at the same time to perform a Diffie-Hellman-Merkle style key exchange.

[1] en.Wikipedia.org/w/index.php?...nge.svg&page=1

[2] For a video overview of this process, see http://www.youtube.com/watch?v=YEBfamv-_do

[3] Sometime, instead of seeing 17 mod 5 = 2, you’ll see 17 ≡ 2 (mod 5). The ≡ symbol means “congruent to” and means that 17 and 2 are equivalent, after you consider the modulus 5.

[4] Many details have been left out, including how e and d are determined, and why this all works. For a bit more detail, see http://www.youtube.com/watch?v=wXB-V_Keiu8, or http://doctrina.org/How-RSA-Works-With-Examples.html

Adapted from Math in Society by David Lippman, hosted on LibreTexts (math.libretexts.org) and licensed under CC BY-SA 3.0. Changes were made. License: CC-BY-SA-3.0.

These eBooks are a prerelease and are not yet certified conformant with WCAG 2.1 AA or ADA Title II. Every page is built against an automated accessibility gate, and the published editions will meet ADA Title II requirements when they release in late September 2026. If something is unusable, please tell us.