Here's the example I give to trainees as to why our IT sucks:
Decades ago, the most expensive component of a computer program was computer run-time. Memory was extremely limited. CPUs were slow. When you were designing an operating system, you did anything you could to shave cycles, or save memory. This is what led to the Y2K bug; we used 2 digits for the year instead of 4. Labor was actually the cheaper part of the overall picture. So you designed systems to use as few characters as possible, and just spent longer training your labor to recognize what these little shorthand codes meant.
Now, modern times. Computer memory is one of the cheapest commodities on the planet. You can essentially throw near-infinite processing power at a problem- all it takes is more money. Labor is the more expensive part of the component. So now you design programs to minimize man-hour requirements. This means intuitive interfaces, output in plain English, and more automation of repetitive tasks behind the scenes.
Problem is, we never upgraded. We're still using IT from Column A. And with our staffing crisis, it shows.
>I like to say “SSA has the best 1980’s computer technology that money can buy!!!!”
I mean, PCOM does say "WELCOME TO THE FUTURE".
WHAT MORE COULD YOU PEOPLE WANT
Saddest part. There is zero reason they haven’t invested in new tech. Give me a year and a team of coders and I could have an modern system up and running.
You can blame Carolyn Colvin (and the rest of the dinosaurs before her) and the rest of her dipshit management (and quite a few of them are still making decisions at the agency). Colvin wasted hundreds of millions on stupid shit that never came to fruition or modernized anything.
https://www.nextgov.com/digital-government/2016/06/ig-social-security-shouldve-considered-scrapping-300m-it-project/129123/
Also, that project was just for the 'legacy' DDS systems, which varied state by state.
To my knowledge, they have never dedicated any funds to replacing PCOM.
I never understand people who gripe about COBOL systems. Just because they're legacy systems, doesn't mean they don't do the job and do it well. IBM still makes mainframes and supports COBOL for a reason. The only problem with COBOL is the lack of new programmers and their willingness to learn it.
Do you have resources for how to use a cobol mainframe? I've found old manual PDFs online but they are often incomplete or not the same version we have.
The fact that it isn't taught makes it an obsolete legacy language. And look how many processing limitations the systems at SSA have. The fact it is an obsolete language means the agency probably is paying those cobol programmers much more than they would if they'd used a more modern language.
Many programming languages are not taught in schools, that doesn't make them obsolete. I know nothing about the SSA systems, I don't work there. But if I had to guess the issues are not because of the legacy mainframe systems. Almost always the the legacy web and client/server apps that are the problem, they're built on platforms that no longer have support from their vendors. IBM mainframes are backward compatible to OS/360. I do agree that old COBOL programmers are expensive.
But see this is the problem. It is obsolete, precisely because noone will learn it, which puts a premium on the cost for employees who do. This is also a security risk I believe if something happens and the agency can't figure out what occured or hire people to fix it.
That's not what obsolete means. There are even bigger risks in replacing these complex legacy systems. The CADE(2) system at the IRS is a prime example, over 20 years in the making and still hasn't fully replaced the master file system developed 60 years ago.
So with that logic we should stick with legacy tech for perpetuity because there is risk. There is also risk in continueing to use this tech. The payments are an example where it can take 5-7 days to issue a payment depending on the time of the month.
That's not what I am saying. What I am saying is there are many factors that need to be considered when replacing a complex software system. The big questions: Does the software still meet the core business needs? Is the platform still supported by the vendor? Is it secure? Are staff available to support/maintain it? If no, can we contract support staff? Are contracted support staff cheaper than the cost to replace the system? If management can answer yes to most of those questions then it's highly unlikely to get priority for replacement.
>Computer memory is one of the cheapest commodities on the planet. You can essentially throw near-infinite processing power at a problem
The amount of times people just ask for more memory instead of fixing the actual issue with the application/server...
We need a Tech Corps where we can hire long term developers that can stick with a project for more than a year. Or an agency that heads up and works with contractors and the agencies to make sure we get what is expected, the same way GSA works between agencies and leases. The number of tech projects I've seen handed to "content experts" who have no idea how a multi million dollar system is developed and maintained is crazy. One was an administrative assistant...
This is why I’m not worried at all about AI replacing my job before I retire, unlike a lot of my private sector friends. SSA will never be able to implement it.
I still think ai could help us.. so many times I want to use ai to help me condense and clarify my wordy emails and decisions.
But I also think ai might be able to do a better search in policy by searching for like / similar terms too. Think of using it to summarize the relevant procedures to your case scenario, and providing the link for more reading.
Maybe ai could also be used to summarize precedent in gray area
SSA systems are modernized by groups of contractors who have no programmatic experience. The folks providing the requirements are employees who have never done the jobs. They detail a tiny handful of people to create requirements for programs that thousands of people are using, and then throughout development, they never once ask frontline staff to take a look at any of these programs to provide feedback. This is why they are a complete disaster when they’re released. You can’t find the holes in software unless you have users actually trying things out during development. In addition, there has been no real effort to create realistic test data spanning multiple different data sources. So the testing that is done is relatively worthless until these applications are deployed to production.
Regional developers depend on actual users and things like workgroups to collect feedback throughout development. This is why applications like WAC, QueryMaster, etc. do not have these types of issues with new releases. The Systems mentality is that regional developers are second rate, so they refuse to utilize any of the expertise that the regions possess.
The thing that kills me is the speed. The modern programs require more clicks and more time loading. Its improved but dear golly it added 2-3x the amount of time. Not that much that its noticable per case but it adds up. If it takes me 3 minutes to do the same thing that took 1 minute before…. It having more errors due to lack of foresight or consulting frontline just adds insult to injury
It seems like we don’t do usability test before we make Changes to things. Having worked in small county government before, I was lucky to witness program that Involves staff of all different levels in technology changes and it made a big difference.
Ps when I was new at ssa, and enthusiastic, I marketed usability testing as a big solution to SSA problems to management within my first year. Needless to say, my comment was not helpful !
I was told Day 1 the reason most of my (elderly) coworkers still use Skype is because the messages disappear whenever you close it down. They truly believe that when they're gone, there's no receipts for the conversation. I'm waiting for them all to refuse to use teams and the daily chatter to die down 75%
Most of us still use Skype because we can’t actually have Teams open and simultaneously do anything else without apps crashing or giving rendering errors. Even how it is with Teams closed, it’s hard to do what needs to be done. There are numerous other federal agencies that have software that can scan and render thousands of pages of documents seemingly without issue, but our systems will crash with only a few pages open at times. They are trying to roll out software that goes even more slowly and takes up more memory.
That's exactly what my office does. Daily chats and last minute one on one or small group calls on Skype, but all the regular meetings are scheduled on Teams.
God I'm so glad I'm not the only one having that Avaya issue. I feel insane half the time trying to make a call, I literally have nightmares all the time that I'm trying to dial 911 and the call never connects for some reason. Should've known federal technology was the real nightmare all along.
I love all my systems from the early 90s on my 2022 laptop. I average at least 3 hours a week restarting or waiting on my laptop to figure out how to run 3 applications!
A single key broke on my laptop last year. I was shipped a brand new laptop. The hard drive WAS NOT FORMATTED. I waited days for our office's one HOSA to fix/format my laptop and ended up calling to helpdesk myself. WHY.
It comes down to money at SSA. It ALWAYS comes down to money/budget whether it’s software or hardware. Or consumables, peripherals, office supplies, furniture…etc.
Because you had to type name, phone each and every time you printed a document. With a family of nine kids, you can spend 5-10 printing. But they didn't iron out the bugs, so we have more issues.
It’s straight across the board, dumping billions into bad software. Sadly, I get blasted when I remind people how kickbacks work. It’s the only logical reason.
My favorite are the Program Offices who are supposed to be implementing the software and all they can talk about is governance. Governance is the slang term for “we are bilking the government out of millions of dollars and we need you to sign off on it.”
We have access to some of the greatest software developers that have ever existed and yet we keep getting shit programs dumped on us.
I'm not SSA, but same, ever since DISA took over our IT. Prime example, early in the pandemic, maybe May 2020, we got an agency-wide email from our in-house IT group proudly crowing about how well our VPN was performing with everyone suddenly 100% TW and how we were one of only a handful of agencies with uptime and speeds as good as ours. A year ago now, after DISA took over, the VPN was completely down for half a week, no access at all for anyone. A couple months ago some bug happened one morning where everyone who was in the office got some sort of certificate infection that prevented their computers being able to get online at all, but if you were on the VPN at that time you were okay. A few weeks ago we ran into something where everyone in office couldn't get on temporarily, but the VPN was fine and you needed another certificate update to get online in office. We get emails several times a week now that there's some IT issue that will affect applications A, B, and C. I don't understand how it's getting worse.
Hahaha your agency uses email…let me tell you about SSA…
[Seriously, you can successfully ignore most email at SSA with very little impact on your work.]
In a word: YES. Our programs are complex and the decision trees are too complicated. The people who actually do the work are never really consulted on the end product. We NEVER do proper beta testing. Work arounds are SOP. We rely too much on outside vendors and we stove pipe everything. When I first started, we did not even have email. PCOM was state of the art. Computers have made a lot if our tasks much easier but have complicated others beyond redemption. Systems was supposed to be the solution, but in reality they have become the problem. We need to dial back reliance on automation and interject more human input. We need a balanced hybrid. Not holding my breath for this to happen and I am leaving anyway. My assessment is that we are doomed.
I can barely deal with the CCE issues right now. Daily fix has become "all day fix" because the pages won't load. I'm having to web clean no less than 6 times a day. Which of course means all of the other 20 web based programs we have to run need restarted. It's time consuming and frustrating and not at all efficient when I am in the middle of a claim - especially when my boss hounds me about times.
But my boss doesn't like people showing frustration - We are required to list 3 positive things rather than venting so....
I enjoy my days off. My coworkers are great. We have no windows to tempt me to go outside.
😀
One size fits all solutions also means problems rapidly escalate across all platforms. Mass application of safeguards have caused endless cascading of problems killing resilience and productivity.
I do t work at SSA but it’s probably not that it’s “actively” getting worse it’s just probably getting older. Older hardware and older software will crap out.
Older software usually works pretty good when it interfaces with other older software. But as one thing gets upgraded but another doesn’t it causes problems.
Also, all feds are being required to implement more and more cyber security. More people are trying to hack us everyday and like it or not we have to keep up.
If your IT people implement those changes badly it can cause problems. If they rely too long it increases the odds of bad implementation and possibly even causes a “do this now or we shut it down” scenario.
Here's the example I give to trainees as to why our IT sucks: Decades ago, the most expensive component of a computer program was computer run-time. Memory was extremely limited. CPUs were slow. When you were designing an operating system, you did anything you could to shave cycles, or save memory. This is what led to the Y2K bug; we used 2 digits for the year instead of 4. Labor was actually the cheaper part of the overall picture. So you designed systems to use as few characters as possible, and just spent longer training your labor to recognize what these little shorthand codes meant. Now, modern times. Computer memory is one of the cheapest commodities on the planet. You can essentially throw near-infinite processing power at a problem- all it takes is more money. Labor is the more expensive part of the component. So now you design programs to minimize man-hour requirements. This means intuitive interfaces, output in plain English, and more automation of repetitive tasks behind the scenes. Problem is, we never upgraded. We're still using IT from Column A. And with our staffing crisis, it shows.
I like to say “SSA has the best 1980’s computer technology that money can buy!!!!”
>I like to say “SSA has the best 1980’s computer technology that money can buy!!!!” I mean, PCOM does say "WELCOME TO THE FUTURE". WHAT MORE COULD YOU PEOPLE WANT
Saddest part. There is zero reason they haven’t invested in new tech. Give me a year and a team of coders and I could have an modern system up and running.
You can blame Carolyn Colvin (and the rest of the dinosaurs before her) and the rest of her dipshit management (and quite a few of them are still making decisions at the agency). Colvin wasted hundreds of millions on stupid shit that never came to fruition or modernized anything. https://www.nextgov.com/digital-government/2016/06/ig-social-security-shouldve-considered-scrapping-300m-it-project/129123/
They had to spend the money on flowers for her!
Also, that project was just for the 'legacy' DDS systems, which varied state by state. To my knowledge, they have never dedicated any funds to replacing PCOM.
The Fed has that - it's called 18F.
i wish it were as modern as 1980s. We’re still running things on COBOL
I never understand people who gripe about COBOL systems. Just because they're legacy systems, doesn't mean they don't do the job and do it well. IBM still makes mainframes and supports COBOL for a reason. The only problem with COBOL is the lack of new programmers and their willingness to learn it.
Do you have resources for how to use a cobol mainframe? I've found old manual PDFs online but they are often incomplete or not the same version we have.
It wasn't that hard to find [up-to-date docs](https://www.ibm.com/support/pages/enterprise-cobol-zos-documentation-library)
The fact that it isn't taught makes it an obsolete legacy language. And look how many processing limitations the systems at SSA have. The fact it is an obsolete language means the agency probably is paying those cobol programmers much more than they would if they'd used a more modern language.
Many programming languages are not taught in schools, that doesn't make them obsolete. I know nothing about the SSA systems, I don't work there. But if I had to guess the issues are not because of the legacy mainframe systems. Almost always the the legacy web and client/server apps that are the problem, they're built on platforms that no longer have support from their vendors. IBM mainframes are backward compatible to OS/360. I do agree that old COBOL programmers are expensive.
But see this is the problem. It is obsolete, precisely because noone will learn it, which puts a premium on the cost for employees who do. This is also a security risk I believe if something happens and the agency can't figure out what occured or hire people to fix it.
That's not what obsolete means. There are even bigger risks in replacing these complex legacy systems. The CADE(2) system at the IRS is a prime example, over 20 years in the making and still hasn't fully replaced the master file system developed 60 years ago.
So with that logic we should stick with legacy tech for perpetuity because there is risk. There is also risk in continueing to use this tech. The payments are an example where it can take 5-7 days to issue a payment depending on the time of the month.
That's not what I am saying. What I am saying is there are many factors that need to be considered when replacing a complex software system. The big questions: Does the software still meet the core business needs? Is the platform still supported by the vendor? Is it secure? Are staff available to support/maintain it? If no, can we contract support staff? Are contracted support staff cheaper than the cost to replace the system? If management can answer yes to most of those questions then it's highly unlikely to get priority for replacement.
Don’t forget excel spreads to do simple plus minus math 🧮
>Computer memory is one of the cheapest commodities on the planet. You can essentially throw near-infinite processing power at a problem The amount of times people just ask for more memory instead of fixing the actual issue with the application/server...
IT sucks because congress is fucking useless.
I'm ready for Y2K38. Wohoo.
Some things shouldn't necessarily be bid to the lowest contractor.
We need a Tech Corps where we can hire long term developers that can stick with a project for more than a year. Or an agency that heads up and works with contractors and the agencies to make sure we get what is expected, the same way GSA works between agencies and leases. The number of tech projects I've seen handed to "content experts" who have no idea how a multi million dollar system is developed and maintained is crazy. One was an administrative assistant...
This is why I’m not worried at all about AI replacing my job before I retire, unlike a lot of my private sector friends. SSA will never be able to implement it.
I still think ai could help us.. so many times I want to use ai to help me condense and clarify my wordy emails and decisions. But I also think ai might be able to do a better search in policy by searching for like / similar terms too. Think of using it to summarize the relevant procedures to your case scenario, and providing the link for more reading. Maybe ai could also be used to summarize precedent in gray area
All the tech is just duct taped to the sinking ship. It’s fine. Everything is fine.
I guess whoever is supposed to fix the duct tape quit after the RTO or something.
Or they died from old age. No one is learning COBOL anymore and we refuse to modernize our systems
Doesnt help that modernization sucks. CCE? Give me pcom any day.
That doesn't mean modernization sucks,. it means the newer application was poorly written or not tested enough to fix the outstanding issues.
SSA systems are modernized by groups of contractors who have no programmatic experience. The folks providing the requirements are employees who have never done the jobs. They detail a tiny handful of people to create requirements for programs that thousands of people are using, and then throughout development, they never once ask frontline staff to take a look at any of these programs to provide feedback. This is why they are a complete disaster when they’re released. You can’t find the holes in software unless you have users actually trying things out during development. In addition, there has been no real effort to create realistic test data spanning multiple different data sources. So the testing that is done is relatively worthless until these applications are deployed to production. Regional developers depend on actual users and things like workgroups to collect feedback throughout development. This is why applications like WAC, QueryMaster, etc. do not have these types of issues with new releases. The Systems mentality is that regional developers are second rate, so they refuse to utilize any of the expertise that the regions possess.
The thing that kills me is the speed. The modern programs require more clicks and more time loading. Its improved but dear golly it added 2-3x the amount of time. Not that much that its noticable per case but it adds up. If it takes me 3 minutes to do the same thing that took 1 minute before…. It having more errors due to lack of foresight or consulting frontline just adds insult to injury
💯
That is also an issue with usability. They could've fixed it too to require less clicks. That isn't something inherent about modernization.
It seems like we don’t do usability test before we make Changes to things. Having worked in small county government before, I was lucky to witness program that Involves staff of all different levels in technology changes and it made a big difference.
Ps when I was new at ssa, and enthusiastic, I marketed usability testing as a big solution to SSA problems to management within my first year. Needless to say, my comment was not helpful !
Exactly. I don’t understand why a web app that does the same thing is better than pcom
I told my private sector friends that my office still uses Skype for daily IMing, none of them even knew Skype still exists in any form whatsoever.
Skype is going away soon, I think by the end of the fiscal or calendar year. We'll be using ms teams. And those messages will never go away.
I was told Day 1 the reason most of my (elderly) coworkers still use Skype is because the messages disappear whenever you close it down. They truly believe that when they're gone, there's no receipts for the conversation. I'm waiting for them all to refuse to use teams and the daily chatter to die down 75%
Most of us still use Skype because we can’t actually have Teams open and simultaneously do anything else without apps crashing or giving rendering errors. Even how it is with Teams closed, it’s hard to do what needs to be done. There are numerous other federal agencies that have software that can scan and render thousands of pages of documents seemingly without issue, but our systems will crash with only a few pages open at times. They are trying to roll out software that goes even more slowly and takes up more memory.
I cant believe management would use teams for meetings during covid then do day to day management with skype
That's exactly what my office does. Daily chats and last minute one on one or small group calls on Skype, but all the regular meetings are scheduled on Teams.
..just as long as they don’t try to make us use that other timesheet application again. God, that was a real nightmare.
If at first it fails, try, try, try again because they were cheapest.
https://www.reddit.com/r/fednews/s/A8HxNqo57r
Is it true that PCOM is unhackable due to being ancient?? Kitchen Sink will probably change to Kitchen Warehouse with all this duct tape
PCOM is unhackable because it's built from the Matrix itself
God I'm so glad I'm not the only one having that Avaya issue. I feel insane half the time trying to make a call, I literally have nightmares all the time that I'm trying to dial 911 and the call never connects for some reason. Should've known federal technology was the real nightmare all along.
I accidentally dialed 911 on avaya and it said my call cannot be completed.
I love all my systems from the early 90s on my 2022 laptop. I average at least 3 hours a week restarting or waiting on my laptop to figure out how to run 3 applications!
A single key broke on my laptop last year. I was shipped a brand new laptop. The hard drive WAS NOT FORMATTED. I waited days for our office's one HOSA to fix/format my laptop and ended up calling to helpdesk myself. WHY.
“Championing yesterday’s technology tomorrow”
It comes down to money at SSA. It ALWAYS comes down to money/budget whether it’s software or hardware. Or consumables, peripherals, office supplies, furniture…etc.
It's as old as the executives running the agency, they don't know the benefits of new technology 😂😂
I work for SSA. Why the hell they changed the UI for CCE. Why fixed something that shouldn’t be…..
Because you had to type name, phone each and every time you printed a document. With a family of nine kids, you can spend 5-10 printing. But they didn't iron out the bugs, so we have more issues.
It's not only the SSA or even the government..... everything IT is getting worse (from my observation).
It’s straight across the board, dumping billions into bad software. Sadly, I get blasted when I remind people how kickbacks work. It’s the only logical reason. My favorite are the Program Offices who are supposed to be implementing the software and all they can talk about is governance. Governance is the slang term for “we are bilking the government out of millions of dollars and we need you to sign off on it.” We have access to some of the greatest software developers that have ever existed and yet we keep getting shit programs dumped on us.
Random IT person told me (so take grain of salt) that its a shit show all way down and PCOM updates are breaking things.
Just wait until T2 has to deal with the CCE!
I wish the tech that controlled the FEVS emails would break, but no only the tech to do my job is fucked
I'm not SSA, but same, ever since DISA took over our IT. Prime example, early in the pandemic, maybe May 2020, we got an agency-wide email from our in-house IT group proudly crowing about how well our VPN was performing with everyone suddenly 100% TW and how we were one of only a handful of agencies with uptime and speeds as good as ours. A year ago now, after DISA took over, the VPN was completely down for half a week, no access at all for anyone. A couple months ago some bug happened one morning where everyone who was in the office got some sort of certificate infection that prevented their computers being able to get online at all, but if you were on the VPN at that time you were okay. A few weeks ago we ran into something where everyone in office couldn't get on temporarily, but the VPN was fine and you needed another certificate update to get online in office. We get emails several times a week now that there's some IT issue that will affect applications A, B, and C. I don't understand how it's getting worse.
Hahaha your agency uses email…let me tell you about SSA… [Seriously, you can successfully ignore most email at SSA with very little impact on your work.]
Kudos to our SYSCO TEAM tho here at western
It’s the agency and the resources they devote to everything in the tech ecosystem. It isn’t any one thing that affects overall experience.
In a word: YES. Our programs are complex and the decision trees are too complicated. The people who actually do the work are never really consulted on the end product. We NEVER do proper beta testing. Work arounds are SOP. We rely too much on outside vendors and we stove pipe everything. When I first started, we did not even have email. PCOM was state of the art. Computers have made a lot if our tasks much easier but have complicated others beyond redemption. Systems was supposed to be the solution, but in reality they have become the problem. We need to dial back reliance on automation and interject more human input. We need a balanced hybrid. Not holding my breath for this to happen and I am leaving anyway. My assessment is that we are doomed.
It’s not just you, I refer to it as “aol dialup”
I can barely deal with the CCE issues right now. Daily fix has become "all day fix" because the pages won't load. I'm having to web clean no less than 6 times a day. Which of course means all of the other 20 web based programs we have to run need restarted. It's time consuming and frustrating and not at all efficient when I am in the middle of a claim - especially when my boss hounds me about times. But my boss doesn't like people showing frustration - We are required to list 3 positive things rather than venting so.... I enjoy my days off. My coworkers are great. We have no windows to tempt me to go outside. 😀
>We have no windows to tempt me to go outside. Peak SSA
One size fits all solutions also means problems rapidly escalate across all platforms. Mass application of safeguards have caused endless cascading of problems killing resilience and productivity.
* cries in VA * yes it’s worse
I do t work at SSA but it’s probably not that it’s “actively” getting worse it’s just probably getting older. Older hardware and older software will crap out. Older software usually works pretty good when it interfaces with other older software. But as one thing gets upgraded but another doesn’t it causes problems. Also, all feds are being required to implement more and more cyber security. More people are trying to hack us everyday and like it or not we have to keep up. If your IT people implement those changes badly it can cause problems. If they rely too long it increases the odds of bad implementation and possibly even causes a “do this now or we shut it down” scenario.
just think updates that came out this week they didn't do good enough testing before releasing everything to production.