Notification mails don't arrive anymore

Found an issue with the phpBB system here at NESdev? Use this forum to report problems.

Moderator: Moderators

Posts: 10241
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: Notification mails don't arrive anymore

Post by lidnariq » Sun Jan 20, 2019 12:07 pm

Memblers wrote:For the past week it has been emailing through, but I'm not sure I have the SPF record set properly for, and it probably needs to be changed on as well, I'll need WhoaMan to do that.
Yeah, the domain that it is reportedly coming from has to say what mail servers are allowed to send mail on its behalf. And "mx" doesn't take a parameter...

User avatar
Posts: 4218
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Notification mails don't arrive anymore

Post by koitsu » Sun Jan 20, 2019 4:02 pm

That would be an SPF DNS record.

User avatar
Posts: 2044
Joined: Sat Sep 07, 2013 2:59 pm

Re: Notification mails don't arrive anymore

Post by DRW » Sun Feb 03, 2019 4:19 am

Recently, the notification mails appear at completely random times and out of order. Today, I got a mail for the first reply in a thread, despite having received multiple notifications for the other replies before.

Seriously, there is some major issue in your mailing system.
My game "City Trouble":

User avatar
Posts: 48
Joined: Sun Apr 08, 2018 11:45 pm
Location: Southern California

Re: Notification mails don't arrive anymore

Post by samophlange » Thu Feb 07, 2019 2:43 pm

A friend of mine is trying to register an account here, but he's not able to get the account activation email. He's tried a few different email addresses, and tried using the feature to re-send the activation email, but is still unsuccessful. Maybe that is related?

Also, if we could get his account authorized while this email issue is ironed out, that would be great! Just PM me for his account details. :)

User avatar
Posts: 4218
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Notification mails don't arrive anymore

Post by koitsu » Tue Feb 26, 2019 8:52 pm

These have been working for me for several days now. Not sure if something changed or not, but thought I'd mention it. These are the actual headers (with some stuff modified, and a couple initial lines removed), to give folks some idea of what the mail flow is like:

Code: Select all

From  Tue Feb 26 18:10:01 2019
Return-Path: <>
Received: from ( [])
        by (Postfix) with ESMTP id 30E8B4E7F0
        for <XXX>; Tue, 26 Feb 2019 18:05:40 -0800 (PST)
Received: from (unknown [])
        by (Postfix) with ESMTP id C2638A034F
        for <XXX>; Tue, 26 Feb 2019 19:05:38 -0700 (MST)
Received: from ([])
        by cmsmtp with ESMTP
        id yobKgpppND8RpyobKgDPU1; Tue, 26 Feb 2019 19:05:38 -0700
X-Authority-Reason: nr=8
X-Authority-Reason: s=1
X-Authority-Analysis: $(_cmae_reason
Received: from [] (port=35686
        by with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256)
        (Exim 4.91)
        (envelope-from <>)
        id 1gyobK-000zYs-E3
        for XXX; Tue, 26 Feb 2019 19:05:38 -0700
Subject: XXX
From: <>
Reply-To: <>
Sender: <>
MIME-Version: 1.0
Message-ID: XXX
Date: Tue, 26 Feb 2019 19:04:02 -0700
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: phpBB3
X-MimeOLE: phpBB3
X-phpBB-Origin: phpbb://
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname -
X-AntiAbuse: Original Domain -
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain -
X-BWhitelist: no
X-Source-L: No
X-Exim-ID: 1gyobK-000zYs-E3
X-Source-Sender: ( []:35686
X-Email-Count: 3
X-Source-Cap: bWVtYmxlcmk7bWVtYmxlcmk7Ym94NjA5LmJsdWVob3N0LmNvbQ==
X-Local-Domain: no
Status: RO
Content-Length: 514
Lines: 17
Running full headers through a tool like this sheds some light on the topology and possible causes. The topology currently is this:

1. (nesdev box) --> (who is a general web/hosting provider); could be a service/company/thing running on BlueHost
2. --> something I can't exactly figure out, maybe local mail parsing of some kind, or some TCP/SSL tunnel, I don't know.
3. Somehow at this point the mail makes it to
4. unifiedlayer sends the mail along to another mail server they themselves run
5. That server then sends it out onto the Internet (in this case, my own server that I use to manage my own Email)

I have no idea what the relationship is between all these things, but no wonder there are problems. Things could go missing, flagged, whatever -- and nobody has any insights into why because there appear to be 2 "intermediate" parties/providers involved (BlueHost and UnifiedLayer). That said, using said tool, we find something interesting:

During step #5, the tool reports "Blacklisted". I was like, "what? Surely not my box". Nope, not mine, but rather UnitedLayer! ... ailheaders

They seem to be on several major/key DNSBL lists, which means they have a history of sending out spam. It doesn't mean nesdev is sending out spam -- they aren't -- it means that one of those intermediary servers does. Anyone running a mail server on the Internet that does DNSBL lookups and sees the IP of UnitedLayer mail servers ( in this particular case, but I imagine they have tons) could have their email flat out rejected (i.e. you never get a copy), or possibly accepted but scored as spam (i.e. the mail arrives but in your spam folder).

This is from a SORBS DNSBL lookup:
This is really not good, guys.

Finally: -- the sending domain -- has no SPF (TXT) records of what SMTP servers are authoritative for sending mail out on its behalf. This isn't the reason for the above, I assure you, but it is certainly complicating matters too. In fact, maybe it lacks SPF/TXT records *because* of this whole BlueHost->UL thing, I don't know.

Code: Select all

$ dig txt

; <<>> DiG 9.12.3-P4 <<>> txt
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31516
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

; EDNS: version: 0, flags:; udp: 4096
; COOKIE: d8b654e3b3aafbb92633df695c7608e5370799386a39067d (good)
;                    IN      TXT

;; AUTHORITY SECTION:             10735   IN      SOA 2018091600 17404 1800 1814400 14400

;; Query time: 0 msec
;; WHEN: Tue Feb 26 19:49:57 PST 2019
;; MSG SIZE  rcvd: 131
Not bragging or egoing, I assure you, but Parodius did not work this way. When nesdev was hosted with me, the mail went directly from our box out onto the Internet. I ran a tight ship and did not use intermediary mail providers -- because all it takes is one customer sending spam via such a place and suddenly that mail provider is blacklisted. There's a lot of good reasons I did everything directly.

I kinda wonder if the community would actually prefer to have the nesdev sites hosted back on something that isn't set up this way, i.e. done the "old way" (without ads too) and with reliable Email and so on. I could probably do that in a matter of days, though more likely weeks given my full-time job and so on. I just have a feeling WhoaMan is full-time swamped with his job, runs this as a hobby that "mostly works", but when it breaks/misbehaves/issues like this happen, it's hard to get them fixed (given the model/topology).

Posts: 3
Joined: Fri Dec 22, 2017 5:42 am

Re: Notification mails don't arrive anymore

Post by ArtemisK » Wed Feb 27, 2019 10:51 am

koitsu wrote:Changing the From: line, or the SMTP envelope From, will not fix this problem.
Do not be so categorical.
Note: SMTP was originally defined by two separate documents, RFC-821, which defined the communication protocol between sending and receiving servers, and RFC-822. I fix it by using free software - that includes the function of "fast fix" SMTP-service and of all protocols. Work for me.
Last edited by ArtemisK on Thu Feb 28, 2019 7:05 am, edited 1 time in total.

User avatar
Posts: 4218
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Notification mails don't arrive anymore

Post by koitsu » Wed Feb 27, 2019 2:19 pm

ArtemisK wrote:Do not be so categorical.
Note: SMTP was originally defined by two separate documents, RFC-821, which defined the communication protocol between sending and receiving servers, and RFC-822. I fix it by using free software - that includes the function of "fast fix" SMTP-service and of all protocols. Work for me.
I have literally no idea what this means (free software that uses the function of "fast fix" SMTP-service and of all protocols) -- this sounds almost like marketing material.

And I'm chuckling at someone throwing RFCs at me as if I don't actually have familiarity with them for 30 years. :-) I might point you to RFC 5321, for example, for a more present-day version of how SMTP behaves, and also point you to RFCs 2505, 5322, 4871, 6376, and 7208 (or 4408 for some history) (re: spam related).

Rephrased: if you think From/From: dictate what will/won't pass spam filters or DNSBL, then that's false. Those are involved, yes, but it's not just one thing. Even SPF/DKIM records aren't responsible for permit/deny or entire scoring. Here's the part a lot of people don't seem to realise:

A lot of spam software analyses Received: and if upon seeing an IP address from a particular CIDR (ex. for Comcast) sending mail through a non-Comcast SMTP server, will flag it and/or score it negatively. It's why you can't just pick a separate SMTP relay host (sometimes called a "smart host") somewhere on the Internet and have mail no longer be flagged/scored negatively or even blocked -- you literally have to send mail through your ISP's mail server otherwise this happens. AWS IP space, for example, is often flagged this way if the mail isn't siphoned through AWS's SES service; Gsuite/Gmail is another one (though some of the big hosting provides like this actually rewrite Received: headers to help with this problem, which is both good and bad depending on how you see it). This is all stuff we (as people) have no control over -- sometimes it's in open-source software, other times it's in large-scale enterprise software (ex. Proofpoint). In either case, there's nothing someone can do about it -- that someone has no control over other people's anti-spam mail configurations.

Finally, you might think "use a VPN service!" Sometimes that can work, but you have to be very careful with your SMTP MTA configuration in this case, ensuring that IP addresses used for Internet-bound delivery utilise that of the local VPN tunnel endpoint, while for local mail only things like are used; if there's an intermediary network that's outside of private-allocated CIDR space (10/8, 192.168/16, etc.) that shows up in Received: due to the MTA adding it then you're still subject to the same problem. You might think "well then disable the MTA entirely and just port forward outbound TCP port 25 traffic to whatever IP the VPN service offers for SMTP", which will work, except you now have locally-delivered mail going out onto the Internet -- things like failing cronjobs trying to be delivered to root@yourmachinename, making its way to your VPN provider's SMTP service, which is not what you want (and why running an MTA is a good thing). You just have to be very careful about MTA configuration.

In this case though, I think a big part of this is because UnifiedLayer is flagged for spam in several (at least 6) public DNSBLs. There's nothing nesdev admins can do about that, other than simplify the topology (read: don't use UnifiedLayer, use BlueHost directly -- if they even offer SMTP servers), and/or set up SPF and/or DKIM records denote what IPs *can* send out mail on behalf of

Post Reply