1 |
Greetings, |
2 |
|
3 |
On Friday, 2022-03-18 11:50:33 +0100, I myself wrote: |
4 |
|
5 |
> Nikos, |
6 |
> |
7 |
> On Thursday, 2022-03-17 19:04:04 +0200, you wrote: |
8 |
> |
9 |
> > ... |
10 |
> > http://mmogilvi.users.sourceforge.net/software/oauthbearer.html |
11 |
> > ... |
12 |
> |
13 |
> Really interesting reading. Thanks for the pointer. And also thanks to |
14 |
> the other responders. |
15 |
|
16 |
Matthew Ogilvie's write-up at the above URL was not only interesting but |
17 |
also extremely helpful. It basically covers four points: |
18 |
|
19 |
1. Background information on OAuth 2. |
20 |
|
21 |
2. A how-to about setting up one's Google mail account for OAuth 2 and a |
22 |
Python script (requiring a small configuration file), which gets you |
23 |
your first two OAuth tokens: an "access token" which has to be passed |
24 |
to Google as a password, but which has only a life time of 60 minut- |
25 |
es, and a "refresh token" which is valid until you change your Google |
26 |
password or maybe also until Google decides enough being enough and |
27 |
starts rejecting your refresh requests with "HTTP Error 400: Bad Re- |
28 |
quest" messages which you'll find at the bottom of the Python trace- |
29 |
back from Matthew's script. As long as the refresh token is valid |
30 |
you will have to use this script on a rather regular basis to get new |
31 |
valid access tokens. |
32 |
|
33 |
3. Gentoo-ready patches for "net-mail/fetchmail" up to at least version |
34 |
6.4.13 which however will only support IMAP and will not work in dae- |
35 |
mon mode, thus requiring a "cron" job for fetching mail. |
36 |
|
37 |
Matthew also points out that Gentoo offers a still masked "net-mail/ |
38 |
fetchmail" version 7.0.0_alpha9-r1 which supports OAuth 2, POP3, and |
39 |
daemon mode. |
40 |
|
41 |
4. A description how to tweak "mail-mta/postfix" so it supports OAuth 2. |
42 |
This involves installing a "cyrus-sasl-xoauth2" plugin not available |
43 |
in the Gentoo repository, a complex configuration setup for a full- |
44 |
fledged MTA which above all is dynamically changing (at least, if you |
45 |
want to handle outgoing mail for more than one user), special "sudo- |
46 |
ers" rules for these mailing users, and "cron" jobs to periodically |
47 |
send their mail off to their Google accounts. |
48 |
|
49 |
Following part 2 of this guide I soon had my Google account OAuth-ready. |
50 |
The Python script from Matthew requires a configuration file the path to |
51 |
which has to be explicitly specified in every call. My configuration |
52 |
file resides at "~/.../oauth.cfg" and contains: |
53 |
|
54 |
client_id=... |
55 |
client_secret=... |
56 |
access_token_file=/home/rainer/.../oauth-access-token |
57 |
refresh_token_file=/home/rainer/.../oauth-refresh-token |
58 |
max_age_sec=1800 |
59 |
|
60 |
The first two lines specify the Google project just created, the next |
61 |
two lines define the absolute paths to the two token files to be used, |
62 |
and the last line sets the access token age before which the token will |
63 |
not be renewed. DO NOT try to add empty or comment lines to this file! |
64 |
|
65 |
Regarding the adaption of "fetchmail" in part 3 I opted for installing |
66 |
version 7.0.0_alpha9-r1, since I was used to POP3 and daemon mode [1]. |
67 |
Version 7 is still masked, so to be able to emerge it execute (in the |
68 |
first "echo" command replace "amd64" with YOUR architecture): |
69 |
|
70 |
# mkdir -p /etc/portage/package.accept_keywords |
71 |
# echo '>=net-mail/fetchmail-7.0.0_alpha9-r1 ~amd64' \ |
72 |
>> /etc/portage/package.accept_keywords/Oauth |
73 |
# mkdir -p /etc/portage/package.unmask |
74 |
# echo '>=net-mail/fetchmail-7.0.0_alpha9-r1' \ |
75 |
>> /etc/portage/package.unmask/OAuth |
76 |
|
77 |
[1] If you do not like installing an alpha-version, want to use POP3, |
78 |
but do not insist on daemon mode, I've meanwhile found out that Gen- |
79 |
too also provides a package "net-mail/mpop" from the same author as |
80 |
the MTA I decided to install instead of "mail-mta/postfix". Howev- |
81 |
er, Gentoo-wise this package is almost two years behind due to the |
82 |
lack of a Gentoo maintainer, so you might want to install it direct- |
83 |
ly from |
84 |
|
85 |
https://marlam.de/mpop/ |
86 |
|
87 |
Of course the new "fetchmail" version also introduced new configuration |
88 |
items. Here is my new "~/.fetchmailrc" file (the "authenticate" direct- |
89 |
ive as well as the last three lines are new): |
90 |
|
91 |
set daemon 60 |
92 |
set invisible |
93 |
set no syslog |
94 |
|
95 |
poll pop.gmail.com |
96 |
protocol POP3 |
97 |
service 995 |
98 |
authenticate oauthbearer |
99 |
|
100 |
username "rainer.woitok@×××××.com" |
101 |
dropdelivered |
102 |
fetchall |
103 |
no keep |
104 |
mda "/usr/bin/procmail -pf %F" |
105 |
passwordfile "/home/rainer/.../oauth-access-token" |
106 |
sslmode wrapped |
107 |
sslcertck |
108 |
|
109 |
Even though I run "fetchmail" in daemon mode, it's not running all the |
110 |
time. It's started once when I log in, but my backup script which I use |
111 |
at least once a day will again terminate it before doing anything else, |
112 |
so my mailbox will never change between my last backup and hibernating |
113 |
my laptop. |
114 |
|
115 |
And yes, I almost never shutdown my laptop, but rather suspend or hib- |
116 |
ernate it. Thus I'll possibly need a fresh access token when I manually |
117 |
start "fetchmail", but it's not necessary to repeatedly update it via |
118 |
"cron" while "fetchmail" isn't running at all. The same holds when re- |
119 |
suming from suspension or hibernation: only when "fetchmail" is running |
120 |
the age of the access token should be checked and potentially a new one |
121 |
requested. |
122 |
|
123 |
So I'll have to use Matthew's "oauth.py" script in three ways: |
124 |
|
125 |
1. Running it the first time or starting over to get both, a new refresh |
126 |
token and a new access token. |
127 |
|
128 |
2. Unconditionally checking the age of the access token and optionally |
129 |
requesting a new one, regardless of whether or not "fetchmail" is ex- |
130 |
ecuting. |
131 |
|
132 |
3. Only checking the age of the access token and optionally requesting a |
133 |
new one, if "fetchmail" is really executing. |
134 |
|
135 |
This leads to wrapper script "~/.../oauth.sh" which knows the options it |
136 |
has to pass to script "oauth.py": |
137 |
|
138 |
#! /bin/bash -u |
139 |
# |
140 |
# Script to retrieve new OAuth tokens from Google mail. |
141 |
# |
142 |
# There are two types of OAuth tokens, an "access token" which only has |
143 |
# a life time of 60 minutes and is required for both, fetching mail from |
144 |
# and sending mail via Google mail, and a "refresh token" which is valid |
145 |
# until changing one's Google mail password or until Google decides it |
146 |
# has been used long enough, and is required to get a new "access tok- |
147 |
# en". |
148 |
# |
149 |
# The action performed by this script depends on its first argument: |
150 |
# |
151 |
# "init": Get a (new) refresh token together with a new access token. |
152 |
# This causes an URL to be sent to standard output, which you |
153 |
# have to point your browser to. At Google's mail login page |
154 |
# use your mail address and password to log into your mail ac- |
155 |
# count, agree to accessing your project, copy the authorizat- |
156 |
# ion code then provided, paste it after the text "Enter ver- |
157 |
# ification code:" in the terminal window where this script is |
158 |
# still running, and hit Enter. |
159 |
# |
160 |
# "update": Update the access token, except when it is younger than the |
161 |
# maximum age specified in configuration file "oauth.cfg". |
162 |
# |
163 |
# no arg: Only when "fetchmail" is running update the access token as |
164 |
# if "update" has been specified. |
165 |
|
166 |
dir=$(dirname $(realpath "$0")) # Symlink-free abs path to script dir. |
167 |
cfg="--config_file=$dir/oauth.cfg" # Configuration file option. |
168 |
|
169 |
if [[ "${1-}" = init ]] |
170 |
then "$dir/oauth.py" "$cfg" --obtain_refresh_token_file |
171 |
elif pgrep -u $USER fetchmail > /dev/null || [[ "${1-}" = update ]] |
172 |
then "$dir/oauth.py" "$cfg" --auto_refresh |
173 |
fi |
174 |
|
175 |
Now keeping the above comment regarding "init" in mind I executed: |
176 |
|
177 |
$ ~/.../oauth.sh init |
178 |
|
179 |
to get my first OAuth 2 access tokens. I then created: |
180 |
|
181 |
1. A new "fetchmail" alias (which will also be run upon login): |
182 |
|
183 |
$ alias fetchmail='~/.../oauth.sh update ; fetchmail' |
184 |
|
185 |
2. A "crontab" entry for my own userid (don't forget to add yourself to |
186 |
group "cron"): |
187 |
|
188 |
# When "fetchmail" is started, it will request a new "OAuth" access tok- |
189 |
# en from Google as soon as the current token is older than 30 minutes. |
190 |
# And as long as "fetchmail" is running, this "cron" job, too, will re- |
191 |
# new the "OAuth" access token when it is older than 30 minutes. Since |
192 |
# we start this "cron" job three times an hour, this will result in acc- |
193 |
# ess tokens (except for the first one, which will be refreshed after 30 |
194 |
# to 50 minutes) being refreshed every 40 minutes. This fits well with |
195 |
# the maximum life time of 60 minutes for any access token. |
196 |
|
197 |
11,31,51 * * * * $HOME/.../oauth.sh |
198 |
|
199 |
3. The hook script "/lib64/elogind/system-sleep/oauth-token.sh" for the |
200 |
"elogind" service: |
201 |
|
202 |
#! /bin/bash -u |
203 |
# |
204 |
# When resuming from hibernation or suspension update Google's OAuth to- |
205 |
# ken for user "rainer". |
206 |
# |
207 |
# This script will be called by "elogind" with exactly two arguments: |
208 |
# |
209 |
# $1: Either "pre" or "post". |
210 |
# $2: Either "hibernate", "hybrid-sleep", "suspend", or "suspend-then- |
211 |
# hibernate". |
212 |
# |
213 |
# The "sleep" command is required to give the host some time connecting |
214 |
# the WiFi before running "oauth.sh", and taking this nap in the back- |
215 |
# ground prevents it from delaying "elogind" while firing up the WiFi: |
216 |
|
217 |
[[ "$1" = pre ]] || (sleep 10 ; sudo -u rainer ~rainer/.../oauth.sh) & |
218 |
|
219 |
Now it's time to just start "fetchmail" via its new alias and to catch |
220 |
up with one's incoming mail ... |
221 |
|
222 |
However, I did not follow Matthew's suggestions in part 4 regarding the |
223 |
configuration of "mail-mta/postfix" because the configuration effort for |
224 |
a full-fledged, system wide MTA as well as adding another "ebuild" file |
225 |
to my local overlay seemed like overkill to me. Up to now I was using |
226 |
"ssmtp" for sending mail, and if I remember correctly, the first "s" in |
227 |
this name was referring to "simple". Its configuration file just con- |
228 |
tained eight lines. Looking around I found Gentoo package "mail-mta/ |
229 |
msmtp", which supports OAuth 2 and also a user local configuration file |
230 |
at "~/.msmtprc". Here are the relevant lines from mine: |
231 |
|
232 |
# Name the only account defined here "default", so it overrides the def- |
233 |
# ault account defined in file "/etc/msmtprc": |
234 |
|
235 |
account default |
236 |
|
237 |
auth oauthbearer |
238 |
domain gmail.com |
239 |
host smtp.gmail.com |
240 |
passwordeval cat ~/.../oauth-access-token |
241 |
port 587 |
242 |
protocol smtp |
243 |
tls on |
244 |
tls_certcheck on |
245 |
tls_starttls on |
246 |
user rainer.woitok@×××××.com |
247 |
|
248 |
Now the only thing left to do is telling one's MUA not to pipe outgoing |
249 |
mail directly to the standard "/usr/sbin/sendmail", but rather to script |
250 |
"~/.../msmtp.sh" instead, so we can refresh the OAuth access token, if |
251 |
necessary, before really sending off the mail. Script "~/.../msmtp.sh" |
252 |
contains: |
253 |
|
254 |
#! /usr/bin/bash -u |
255 |
# |
256 |
# Before sending the mail refresh the access token, if necessary, and |
257 |
# since the mail to be sent is provided on standard input, route stand- |
258 |
# ard input around the call to "oauth.sh": |
259 |
|
260 |
{ cat >&3 ; ~/.../oauth.sh update ; } 3>&1 | msmtp "$@" |
261 |
|
262 |
Should you MUA not provide a way for specifying the programme to which |
263 |
outgoing mail is piped, you can simply modify the symbolic link "/usr/ |
264 |
sbin/sendmail" (which has just been set by package "mail-mta/msmtp" to |
265 |
point to "../bin/msmtp"): |
266 |
|
267 |
# cd /usr/sbin |
268 |
# rm sendmail |
269 |
# ln -s ~rainer/.../msmtp.sh sendmail |
270 |
|
271 |
That's all, happy mailing ... :-) |
272 |
|
273 |
Sincerely, |
274 |
Rainer |
275 |
|
276 |
PS: This mail was sent to you using "mail-mta/msmtp" set-up the way de- |
277 |
scribed above ... :-) |