Hope none of you need to visit a dealership anytime soon

[Charley]

System wide outages and how to handle it always reminds me of the Medieval Help Desk

Sponsored

 

[5thranger]

My sources tell me the same group of people that designed 10r80 transmission also designed this system that crashed and burned.

 

[MountainGoat]

Based on what I‘ve read in earlier posts, the issue is likely related to the latest Sync3 update.

Maybe somebody who didn't update Fordpass entered their own oil change and that's what brought the system down.

This is the downside to everything going digital. It is great until it isn't......

And security always takes a back seat until something like this happens. It will be interesting to see how long it is down for.

I wouldn't even say "until". There are constant cyber attacks and they seemingly never get resolved. And these are just a warm up to the main act.

https://www.csis.org/programs/strategic-technologies-program/significant-cyber-incidents

 

[rang19ca]

When I picked up my truck from the dealer on Wednesday, they were having a system wide computer issue. I was able to get my truck, but they could not supply any paperwork.

 

[RedlandRanger]

Maybe somebody who didn't update Fordpass entered their own oil change and that's what brought the system down.


I wouldn't even say "until". There are constant cyber attacks and they seemingly never get resolved. And these are just a warm up to the main act.

https://www.csis.org/programs/strategic-technologies-program/significant-cyber-incidents

Digital system do provide a lot of benefits - but they also have a very bad downside. With paper, you always have a backup (unless you have a fire or flood). With digital systems you can lose EVERYTHING in the blink of an eye if things are done correctly. And since there is no manual system, when they go down, it is extremely disruptive. We've built systems that tell people what they need to do, so they don't need to be trained much. But when those same systems go down, very few people understand how to do work without them.

Effective security CAN be done - it is just very low priority for most places. Most places anymore are more interested in "refreshing" things (see the recent FordPass update) to make it look like things are getting better, than making things more resistant to outages.

Sorry..... I see this every day at work and it is becoming more and more prevalent.

 

[TJC]

This is the downside to everything going digital. It is great until it isn't......

And security always takes a back seat until something like this happens. It will be interesting to see how long it is down for.

When I left IBM for Lenovo, I warned Lenovo not to hire IBM as a supplier of service. They did not heed my advice and suffered 6 outages in the first 6 weeks of occupying our newly constructed HQ in Research Triangle Park, NC!

3000 employees sitting on their hands for days on end during each outage. No network, internet, or phone services. The Lenovo HQ completely offline.

We had the infrastructure and talent on site to house the equipment (I know because I was the I/T project manager overseeing the I/T infrastructure fit up of the 1.3M sq ft complex), but Lenovo chose to place that equipment 10 miles away in an IBM controlled raised floor facility.

This is why you do not farm out your I/T work to a cloud based conglomerate. Your systems are only as good as your weakest link.

Keep it in house. Keep your primary systems air gapped until you need to run a batch update jobs. Keep internet face time (exposure) to a minimum. We are hooked on convenience (corporate America) and do not balance risk vs reward.

I realize most hacks occur due to an individual skirting security guidelines.

I remember an IBM Senior VP reading an "Eyes Only" IBM Confidential document that they moved from a reading room to their laptop. The documents were online, but only the ports in that room were routed to the electronic vault housing these documents. The room was only assessible to those with the highest security clearances.

The executive promptly took a business trip, her car was stolen with laptop in the trunk in the parking garage. Car recovered, but laptop was nowhere to be found. This Senior VP was forced out in days for breaking confidential data handling protocols.

Conglomerates are a single entity with thousands of businesses running on and through their infrastructure. It only takes one individual to skirt secuirty protocols (for whatever reason, but usually convenience) to allow hacker access. Hackers only have to break a single target to access multiple businesses. Once you break through key firewalls, the data stores are ripe for the picking with hacker toolkits..

Secuirty is one of the reasons I run Linux over MS windoz or Apple platforms. It is also why I do not use free cloud services such as Google.

It is also the reason I run the POP eMail protocol instead of IMAP. I understand the convenience, but I also understand the exposures. I don't want my email stored in a cloud account. POP by design, means my mail only traverses the cloud while traveling to my local machine, once on my local system, the eMail is immediately deleted on the cloud server. My eMail client retrieves my eMail every 10 minutes 24/7.

My wife is an I/T security analyst. She keeps me on the straight and narrow, but having spent 30 years involved in all aspects of I/T - computer HW and SW, and in Research Lab Development environments, I already know what she is telling me. I have witnessed the downsides of loose security too often.

 

[Muffin1]

Would this down system effect the Ford Pass App remote start service, i wasn't able to use it a couple of attempts yesterday?

 

[Friday yet?]

Paper is magical. It is called a form and it has five different different sheets, each different color and you just write on the top one and amazingly it transfers to all the others and then you get one color and somebody else can send another color and all goes with this beautiful beautiful room full of a box called filing cabinets.

"Surely you jest" are thinking the young people here....

Thinking about all those forms and filing cabinets and stuff, and wanting to remain on the cutting edge, has me thinking that I may need to go shopping for a new..... pager.

 

[Friday yet?]

My employer has been hacked twice in the past 5 - 6 years. First time was really bad. While I am not an IT guy I seem to remember that the technical term for the actual hack and our response to it was "shit show." We were jacked up for three weeks or so. Longer than that before we were 100% back up and running with all systems. And at year end no less! Those of us that actually get shit done were never told all the details. Rumor has it that it cost us $10M (Self insured to that point) and cost our insurance carrier more on top of that. After that our day to day life became much more of a pain in the ass from an IT perspective. I'm thinking it's pretty much like obtaining the nuclear launch codes now when all I'm trying to do is access a company program to run a report or something.

Second hack? Last year? Different story. Whatever happened, our system/people caught it quickly. Shut everything down. Happened so fast that I can't help but envision sirens going off and some guy in the basement slamming his hand down on a big red button bringing the entire organization to a screeching halt. Communications to the field started almost instantly. Then pretty rapidly various systems were brought back on line one after the other. All in all I'd say that within 4-5 days we were 100% back to normal. So changes were made, some of it a PIA in our daily life, but based on hack #2 I'd say well worth it.

One of our competitors is in the midst of a hack right now. And the admissions they've finally had to make publicly, combined with their employee horror stories posted in various forums online makes it clear that they've had their ass kicked. Can't pay vendors. Can't pay employees, Dead in the water. And we are talking a large international company. Some of their ex-employees work for my assigned dealers now and still have friends "across the street." Was being told this week while at one of my dealers that after talking to their old friends at their previous employer the event is every bit as bad as the online forums lead it to be.

These hacking bastards need to be........ well, I'd better stop. If I speak my mind as to how to deal with them I'll get permanently banned from here for sure. Let's just say I have some ideas and they'd be rather unpleasant for the hackers.

 

[AzScorpion]

"Surely you jest" are thinking the young people here....

Thinking about all those forms and filing cabinets and stuff, and wanting to remain on the cutting edge, has me thinking that I may need to go shopping for a new..... pager.

You don't have to tell me about all those forms and files! Having just moved I went through and shredded a huge 2 draw file cabinet full of folders/files. Most I had to keep for business/tax purposes but I found stuff from (I hate to even admit this) the 90's in there.

One bonus I even found my old Startac flip phone which should work great with that pager. ?

 

[Friday yet?]

I found stuff from (I hate to even admit this) the 90's in there.

Guilty here. If I don't have old stuff stored away, it's simply because the Warden got involved and suggested that I get rid of it.

 

[Friday yet?]

Imagine some of the young'uns on this forum if I told them about my early working days when I kept a roll of quarters in my truck because I had to find a pay phone when I was on the road and needed to call the office....

 

[GhostStrykre]

Car Dealership Guy is covering this story on Twitter as well. Sounds like CDK might be paying the ransom. I think the fear is this could be a protracted affair if they don’t:


Admittedly, several of his posts are auto industry people who are just afraid. The worst case scenario is the best at getting clicks.

Events like these are a good reminder that we, as consumers, need to put some thought into how our data is being managed as we use the products we purchase.

Free apps, your data is the product. I’ve lost count of how many times these damn free apps that I’ve used have been hacked.

Businesses like Target. Target was warned that their security practices were weak, and they decided they didn’t care enough to divert money to fix it. Then they got smoked by a hack.

Google and Apple maintain some high levels of data security. They have to. Otherwise their product credibility will plummet. That being said, that’s why I was always skeptical of “off brand” Android phones. Your phone is the #1 spy device you own. Not something I wanted to leave to the security practices of some rando brand.

and to bring it back to more of an on topic item: GM. Their new vehicles don’t have Android Auto and Apple CarPlay. They paid Google to make a system just for their vehicles. I think it’s pretty obvious that their CEO wanted access to all that juicy data that the phone provides when plugged into the car’s system. They didn’t wanna leave it with Google/Apple. Given GMs track record of peculiar cost cutting, I don’t think I’d ever trust them here. Their infotainment system/data harvester is basically a show stopper for me. Am I paranoid? You bet.

sorry for the long winded post. Just seemed like the moment to discuss this. These are our modern times. You shop for a product with the capabilities you want, but your final checks should ALWAYS include consideration of data security practices and backups. Sadly, this could lead to huge corporations and less competition, but that’s a different topic for a different day.

 

[Chuck Dragisic]

I know what the problem is..... I took my Ranger back to the dealership, This was after they did the blend door drill. It took 3 trips to get the dealership to acknowledge that there was a problem. The wife would complain that the passenger side of the truck was hot...I would take it in and was told that it was not setting a code. It was hot. And I had to pay a diag fee each time. On the third trip, if finally set a code, and they fixed it.
A week after I got the truck back, the A!C began to smell like a gym locker room. I took it back, and told them that in Houston, Texas with all the humidity that we have, it would not generating any condensate. There were no puddles on the driveway, or anywhere. Should be fairly straight forward. It has been sitting at the dealership for a week, with no end in sight.

 

[Racket]

Another Carrington Event would be... fun.

Sponsored