Credit union IT monitoring: a case study in overnight alert texts
A credit union's small IT team had good monitoring, but its alerts landed in a shared mailbox. Overnight job failures could sit unseen until morning. The team sent those alert emails to Welco, and the admin on call now gets the failures as short texts. Like our other customer alert stories, it leaves out names and figures.
7-day free trial, 15 message creditsCard required for verificationNo charge for 7 daysEmail reminder before billing
- Nothing changed on the servers
- The shared mailbox kept every alert
- No per-admin seats
Works by email with
- Microsoft 365 / Outlook
- Backup & batch jobs
Published:
What changed for the admin on call
Failures arrive at the hour they happen
A failed job or a down server reaches the admin on call as a text, not as a message waiting in the inbox.
Nobody watches the inbox overnight
The admins stopped checking the shared mailbox at night on the chance something had failed.
The audit trail is untouched
The shared mailbox still holds every alert, so nothing is lost for audits or the morning review.
No seat to buy for a new admin
Someone new on the on-call bench accepts an email invite and replies YES. That is the whole onboarding.
About the credit union and its overnight jobs
The credit union runs its core system, branch networks and a set of nightly batch jobs. A few admins share on-call duty. Members never see the alerts, but they feel it when a job fails and the morning starts late.
Most of the important work happens overnight. End-of-day posting, file transfers and backups all run while the branches are closed.
Overnight alerts before and after Welco
Nothing changed on the servers. The team changed where the alert emails go.
One shared Outlook mailbox
- Their network monitoring tool watched servers and links, and the job scheduler and IBM i reported on batch jobs. All of them emailed one shared mailbox.
- During the day, someone read that mailbox. At night, an alert was only seen if the on-call admin happened to check.
- A few alerts were forwarded to carrier email-to-text addresses, which became unreliable.
- Large on-call tools offered schedules and phone calls the team didn't need, and charged for every admin as a user.
Failures by text, the rest in email
- Emails that say
DOWN,CRITICALordid NOT completebecome texts. Success notices stay in email. - Overnight alerts go to the admin on call that week, and daytime ones go to the team lead.
- Each text keeps the job name, the result and the time, so it fits on one screen.
- The shared mailbox stays the record, exactly as before.
What a failed IBM i job looks like as a text
A failed overnight job is easy to miss in a busy inbox. Here is the kind of message their IBM i system sends, and the text it becomes.
Alert email from ibmi-jobs@cu.example.org, subject “JOB NIGHTPOST ended abnormally”. Welco trims it to one text of 83 characters: “Welco: [IBM i] Job NIGHTPOST did not complete at 12:47 AM. Reply STOP to opt out.”
Text received: Welco: [IBM i] Job NIGHTPOST did not complete at 12:47 AM. Reply STOP to opt out.
Characters221831 credit
The job name and time are what the admin acts on. The full email stays in the shared mailbox for the morning.
Get overnight job failures on the on-call phone
Send your monitoring and job alerts to Welco. Keep the shared mailbox as your record.
7-day free trial, 15 message creditsCard required for verificationNo charge for 7 daysEmail reminder before billing
How the team set up overnight alert texts
Nothing changed on the servers. The team changed where the alert emails go, and decided which ones deserve a text.
Approve each system that sends alerts
They approved the monitoring server, the job scheduler and the IBM i system as senders. Some alerts still pass through Outlook, so the Microsoft 365 forwarding mailbox was approved too.
Text failures, not successes
Keyword rules text emails that say
DOWN,CRITICAL,completed with errorsordid NOT complete. Routine success notices stay in email. See how batch job alerts are usually set up.Nights to the admin on call
A time rule sends overnight alerts to the admin on call that week. Daytime alerts go to the team lead.
Cut long emails to one text
Job emails carry return codes and log hints. A trimming rule keeps the job name, the result and the time, so each alert fits in one text.
Why some alerts arrived twice at first
Early on, some alerts arrived twice. Welco already sends each email to each phone only once. Here, though, the same alert reached Welco as two separate emails, through a distribution group and a forwarding rule.
The fix was in their own mail setup. Once one of the two paths was removed, each alert produced one text. Duplicate protection handles repeats of the same email, not two different emails.
What sends the alerts
The same pattern fits most bank and credit union IT teams on a small bench.
Frequently asked questions
Did the credit union install anything on its servers or core?
Which overnight alerts did the team choose to text?
DOWN, CRITICAL and did NOT complete trigger a text to that week's on-call admin. Success notices never leave the shared mailbox.Who handles the alerts during business hours?
What would a setup like this cost a credit union?
Let tonight's failed job reach the admin on call
Point your job and monitoring alerts at Welco and keep the inbox as your record.
7-day free trial, 15 message creditsCard required for verificationNo charge for 7 daysEmail reminder before billing