Create a new key pair
First download GPG Suite to create and store your GPG key passphrase (you don't need to download this app to generate a new key pair, but it makes your life easier -- maybe).
Open GPG Keychain and click New on the top to create a new key pair.
Fill out the form and press Create Key to generate one.

Now right click on your newly created key pair and click Copy.

We'll use this copied key to add a new GPG key on GitHub.
Adding a new GPG key
Navigate to your GitHub account -> Setting -> Access -> SSH and GPG keys and click New GPG Key on the bottom.

Now paste your key here and click Add GPG Key and you're ready to go.
Sigining commits
Use -S flag when committing changes:
1$ git commit -S -m "signing commits :)"
If you get a passphrase is unmatched error, you need to tell git which GPG key you're using.
If succeeded, push your commits.
1$ git push
And check your commits from GitHub repo and you'll see a label Verified attached next to your commit.

If it says Unverified, make sure you've added the right GPG keys from the setting.
Tips
If you want every commits to be a signed commit by default, set commit.gpgsign to be true:
1# local 2 $ git config commit.gpgsign true 3 4 # global 5 $ git config --global commit.gpgsign true