Charlie Waters
, ,

When is the “IT Guy” not enough?

Before we start to answer this question, let’s first consider answering when an “IT Guy” (or Gal) IS enough.  Some organizations are adequately served by what the industry calls “Break/Fix” service.  Simply put, when something goes down, you have a resource on speed dial who can come out and get things up and running again.  Many companies use this model successfully or, at least, some variation like perhaps buying a block of hours – this is the same as break/fix except you are buying time in advance and often at a discount.  Perhaps some of your work is still on paper and your processes are mostly manual.  And on the surface, this arrangement is workable (even if not ideal) for businesses who have very simple technology needs.

When Break/Fix Breaks…

At some point, as a business becomes more operationally mature, it begins to leverage technology as a competitive advantage and the underlying technology to drive more efficient workflows becomes more complex.  You are now running servers with key line of business applications that require running a database.  Your workflows are more efficient, your company can scale its efforts, and processes have become automated, reducing human error.  As a result, organizations begin to value its technology operations as being strategic and mission-critical to company success.  As the operational maturity level (OML) continues to increase, an inflection point is reached where the break/fix model no longer works and is a hinderance to efficiency and security.  Your needs have outgrown the old model. 

Think about it this way – the incentive for the break/fix IT guy is misaligned with your organization.  They are rewarded (paid) when your technology is down, not when it stays up and running.  This creates a dynamic where addressing root cause issues of technology failures and building more robust (but also more complex) systems is not in the best interest of the person doing the work.  Why should they invest energy to prevent failures rather than band-aid symptoms or develop workarounds to keep things just stable enough to not get fired?

Yeah, but…

Some IT folks have the integrity to do things in the best interest of their clients despite this not being in their own financial interests.  Unfortunately, I have seen too many of the former and very few of the latter, so the odds are high your “IT Guy” could be taking advantage of you.  Perhaps this isn’t even a conscious decision, but simply the cause and effect of being rewarded to maintain the status-quo.  The other reason is they simply lack the knowledge and experience to manage things in a better way.

Taking a deeper look at the problem…

Let’s examine the usual case:  You hire someone who is inexpensive and eager to grow their skillset.  Their only experience is building a PC or two and setting up the family’s home network.  They know just enough to be dangerous, but they have more knowledge with IT than you do.  You decide to give them a try.  Your company’s network has now become their personal IT playground.  He/she will happily persuade you to try new things in your environment.  Let’s consider when this “new thing” is the backup system for your important documents and company QuickBooks files.  He tells you the system can be implemented with minimal cost because the software is “free” (perhaps Open-Source Linux or something) and you happen to have an old PC that can be repurposed to host the system.  What could go wrong?!

Well, let’s list a few potential issues:

  • Due to inexperience, the tech didn’t ask or know where all the critical data resides and failed to include the QuickBooks files in the backups.  The QuickBooks PC dies and there are no backups.
  • The single drive in the backup server starts failing, but the condition is not known because nothing is monitoring the performance of the hardware.  Your primary application server crashes and there are no good backups (due to the failing hard drive) because your tech never tested restoring the data.
  • A flood in the IT closet destroys both the server and the backup system resulting in total data loss – and there is no off-site copy.
  • One of your employees clicks an email link, unleashing a ransomware attack on your network and because their account had admin privileges on the network, the server and backup files are also encrypted, resulting in total data loss.
  • Your main server crashes and it takes a week for your tech to source new hardware, rebuild the server from scratch, and then restore your data from the backup.  Everything worked as designed, but your tech didn’t consider how long your company could be down while everything was being rebuilt.  Your business just lost a week of productivity.

Truth or Consequences?

We wish we could say these horror stories are complete fabrications, but you would be surprised (or maybe not) at the many ticking time bombs we have come across. To be sure, we won’t name names here, but trust us when we say, “We have seen it all!” The fortunate ones are those who made the switch to more professional IT management before things went south. It would not be a bad idea to quiz “your guy” about what measures are in place to ensure these things don’t happen. You’ll likely get one of two reactions – a smile (with a laundry-list of precautions being taken) or sweat (with a ton of excuses)! You be the judge. And this pop-quiz of sorts doesn’t require being technical…it is easy to read whether someone is confident and knows what they are saying or trying to talk you in circles to avoid answering the question. Reminds me of final exam essay questions where you don’t know the answer, but hope you can write enough to eventually touch on the correct response.

In any of these scenarios, your company will have paid a hefty price for the inexperience of your IT guy.  Important lessons were learned by both parties.  Your business has just become aware of the need to be more operationally mature, and your IT guy knows what not to do next time.  Layer on top of the operational issues, the constantly evolving need for better security, and the problems become even more complex and the risk to your organization that much greater.

Oh, but he is an employee, so there’s more…

Here are some other limitations of having a single resource (perhaps your employee) running IT:

  • Who fixes problems when he/she is on vacation or out sick?
  • Where is the escalation path when issues are outside of your tech’s skillset?
  • You hired them at a low salary, but now they have experience and a resume (at your company’s expense), and they leave you to make 50% higher salary elsewhere.
  • Your IT needs have grown, and you need: a desktop technician, a network/server admin, and an IT Director.  Even if there is overlap in the technical competencies, now you are spending $200K+ (over $16K per month) to hire and retain competent technology staff.

Take note that many of these problems are also inherent in outsourcing IT to a single-guy shop whether the agreement is structured as break/fix (with the problems discussed above) or fixed-fee.

Managed Service Providers to the rescue!

We have looked at why the break/fix model doesn’t work for many organizations AND why hiring IT staff has serious limitations.  There is a sweet spot in Small Business where the MSP model thrives – higher OML organizations who value quality IT services but cannot afford to staff a full IT department.

MSPs operate with fixed-fee monthly services and provide outsourced IT resources for your business.  They staff experienced technology professionals who fill the various roles of an IT department.  MSPs provide best-of-breed tools to monitor and manage your systems, all-you-can-eat help desk support, and even strategic IT management (usually with a virtual CIO service serving as your IT Director).

Managed Service Providers buffer your organization from the challenges of hiring and retaining quality staff, plus provide redundancies in various technical competencies.  MSPs can offer technology talent a better compensation package: 401K, flexible PTO, career tracks with promotions, training programs, performance pay, and other benefits.  And the employee doesn’t have the stress of being on an island with no other technical resources to help when needed.  They are part of a team.

Win-Win

All of this and the MSP’s interests are aligned with the needs of your organization.  A fixed-fee monthly contract means both companies benefit when technology is stable and end-users are productive and happy.  This is a win-win since the cost of outsourcing support is less than staffing an IT department.  So, to answer the question, “When is an IT guy not enough?”: When your organization values the benefits of well-managed technology, but it is not practical to staff your own IT department.

/by
m.a.collins
,

IT Missional Misalignment vs. Strategic IT Investment

Often when working with businesses and organizations that (1) have an IT department, (2) have contracted a third party to act as an IT department, or (3) have the solo “IT Guru” on staff, we find that there is a disconnect between what that IT person/department thinks is their job and what their job really should be.  Most IT personnel think of themselves as virtual firefighters – just keep the existing IT-related equipment and software running.  Certainly, that is an important aspect of their job, but it is far from their primary responsibility (or should be).

Every Organization Today Is an IT-Driven Organization

Think about this for a second – what organization in the 2020’s doesn’t have mission-critical reliance upon IT-related technologies and communications?  Every organization needs Internet access, email and other digital communications, a reliable network infrastructure, and IT devices that provide users consistent and reliable access to these necessary services.  If those IT services and devices malfunction, the entire organization suffers losses from losses in efficiencies, lost opportunities, and potentially monetary losses.  Since every organization is IT-dependent, it’s time that business owners and investors look at IT as foundational to their company’s success, rather than an add-on.

The Short-Sighted IT Firefighter

Many times, IT departments and personnel fall into the day-to-day trap of keeping everything running and mistake that for their mission.  IT personnel further tend to create “job security” through unnecessary complexities and artificially creating reliance upon their personal skills.  This is often in response to a siloed department structure where each department is vying for supremacy and importance.  The departments begin to build walls over time and engage in turf-protection, or even worse, turf wars to protect their departments. 

Nevertheless, no matter what the reason, mistaking firefighting (resolving the day-to-day issues) as the primary mission of IT is incredibly short-sighted.  Having this mindset creates a house of cards over time as IT finds band-aid workarounds to keep things moving along until there are too many balls to juggle and plates to spin. 

Since every organization has as part of its foundation a dependency upon IT, we must rethink our approach to IT.  IT is no longer a convenience – it is a business investment that yields real return.  Imagine the lost opportunities and efficiencies when an organization’s IT house of cards collapses, or the proverbial juggled balls and spinning plates begin to tumble.  You can only cheap your way out in IT for so long before you pay the consequences for doing so.

Strategic IT Investment and Missional Alignment

The IT department (either internal or contracted) needs to see themselves as a critical driver for the missional success of any business or organization.  IT should be represented in organizational strategic planning.  Most large companies can afford a CIO to provide that IT strategic overview.  Smaller companies might get input from their staffed IT person; but IT personnel, while great at problem solving, often have not been exposed to strategic planning initiatives and engineering best practices for IT deployment.  Adding the mindset of trying to keep everyone happy and not spend any more money than they think necessary, and that further complicates their contributions to strategic planning.

 Accordingly, it is often beneficial to employ third party consulting to partner and advise on how to align their dependence upon IT with the mission of the company or organization in a way that is sustainable, secure, and efficient.  At TCS, we call this our vCIO service.  Our mission is to empower organizations and businesses to better serve their client base by using secure and efficient business systems.  One of the ways we do that is by providing insights as how best to approach (and sometimes reboot) their IT goals and approaches moving forward.  We can accomplish that by coming alongside your existing IT personnel and assisting them with mission alignment, or we can completely manage IT as a contracted third-party – whatever and whichever works best for your organization.  Simply put, the business of TCS essentially is to transform IT from a liability into a force multiplier and competitive advantage for respective organizations.  We succeed to the degree that you succeed, and we wouldn’t have it any other way!

/by
Charlie Waters
, ,

Endpoint and Application Security vs The Soft Gooey Middle

Soft, Gooey, What?!

Still no Silver Bullet…

In this article we are going to distinguish between various areas of our defense-in-depth strategy.  If you have read our prior posts, you know security is not a single thing and there is no magic silver bullet, but good security is a combination of layers of defense.  So here is the problem with the traditional approach: much emphasis has been placed on the corporate network with its firewalls, intrusion detection, content filters, hard wired ethernet connections, and encrypted corporate wifi – except there is a paradigm shift toward mobility and this puts our endpoint devices and applications at a disadvantage. 

Safety outside the castle

Access is needed outside of the high castle (corporate) walls where the commoners gather.  Places like Starbucks or the now ubiquitous home office.  These external areas most often do not share the same security features of the traditional workplace network. So, what is “soft and gooey”?  Well, the truth is, even the corporate network is not as secure as we would like to believe.  Yes, it is more secure but with the ever-increasing threats of email phishing, zero-day attacks, and other threats, the constant cat and mouse game of securing the network is often a losing battle.  We still need to address these areas, but even more is needed.  And because of the trend to cut the corporate tether and leverage the advantages of mobility, the current best defense strategy is to assume the corporate network is an unsafe zone and beef up efforts to build security around the endpoint (more and more often a laptop or smart phone/tablet these days) and likewise the application itself.

Endpoint Protection

Not your average antivirus

Endpoint protection is generally reduced to signature-based antivirus.  The flaw is these products are ineffective against new threats that have not yet been cataloged by the software vendor and released as updates.  Also, threats evolve into different variants that are not detected by the antivirus engine and leave your device open to attack.  Installing operating system updates helps but still do not offer protection against unknown vulnerabilities.

More needs to be done.  New “next generation” antivirus products build on the traditional approach by using behavior monitoring and artificial intelligence.  These security products not only block known/cataloged threats but are able to detect unknown threats by looking for malicious behavior by the application running on your device.  Advanced heuristics establish a baseline of “normal” behavior and shuts down activity when a process misbehaves.

Even more is needed

An additional capability involves moving content filtering from the corporate firewall to the endpoint itself.  This can be accomplished with very little additional overhead as the filtering takes place on secure Internet DNS servers (hosted by the security vendor).  This is a valuable security measure when developing a mobility-first strategy.

Who has not seen a VPN commercial these days?!  There seems to be an endless number of companies selling virtual private network technology.  These can be used to extend the corporate network for secure access to on premise and/or cloud-hosted applications. Also, a VPN can be leveraged to encrypt general Internet traffic on an endpoint connected to unsafe/open wireless networks (like Starbucks).

Further, endpoint cloud backup is desirable when there is critical data on a laptop that is not saved frequently to servers. This trend is more common as we rely less on servers and move our data to cloud storage.

Application Protection

C squared = B + HS = I V

Reading a bit like a Phil Mickelson formula to defeat Tiger Woods, the alphabet soup of IT Security can be equally intimidating – we get something like HTTPS+VPN+2FA-MITM = GTG. Much like securing the endpoints in untrusted environments, the applications can be protected from unauthorized access.  Two-factor authentication along with forcing an encrypted connection is a common approach these days.  You will notice most web sites you visit these days use https instead of http.  The former is an encrypted connection while the latter is open to what is called “Man-in-the-middle” (MitM) attacks due to the lack of an encrypted session.  Essentially a hacker can read user passwords and other data sent back and forth over the unencrypted connections while it is much more difficult to do the same thing when the connection is secured using advanced encryption.  Cloud-hosted applications can also use software firewalls to enable many of the same security features traditionally found on the corporate hardware firewalls. 

M365 to the rescue

Microsoft 365 is a good example where application security can be enhanced.  Companies using 365 email have the mail transport encrypted end to end between internal and external parties running on the same Microsoft hosted platform.  Further, anti-phishing and cloud-to-cloud backup can be used to protect the documents and emails stored on the M365 system.  Additionally, Microsoft Teams communications through chat or voice/video calls are encrypted.  There are huge benefits to living within this ecosystem as much as possible as the number of security products needed to protect communication and collaboration can be reduced.  Less complexity also means greater security as there are fewer configurations needed to make the security work.  When combined with Microsoft Azure virtual server hosting, it is now possible to move niche line of business applications and critical company workflows to the secured Microsoft environment.

So What, Now What?

Will legislation force all of our hands?

Responsibility is being placed on the Managed Services Provider to enforce these security measures.  For example, Louisiana Act 117 – Senate Bill 273 requires MSPs that manage infrastructure or end-user systems for “public bodies” to register with the state.  Additionally, MSPs are now required to disclose cyber incidents to the state. 

It is expected for similar legislation to make its way to other states and there will be an increased top-down accountability between regulated organizations and their technology vendors.  This means MSPs will continue to up its security game or be left behind.  Also, managed service providers will become more selective when choosing its clients to ensure there is a closer alignment of operational maturity levels (OML), otherwise there will be constant tension between the MSP obligations versus the organization’s cooperation to improve security.

Follow the leader

The best approach is when a security-minded MSP articulates the reasons behind the need and the client trusts the advice of its technology partner and follows their lead.  For those who refuse to take security seriously, MSPs may be eventually forced to document the opt-out on the client’s part by issuing a legal letter advising of the dangers of not implementing the needed defenses.  This will strain relationships where there is a mismatch of OML or where trust is lacking.  What this means for all of us is the cost of doing business will continue to go up as more products and time will be needed to implement these solutions.  But the risk is too high to ignore the warnings and being wrong about security can result in a higher cost to business due to downtime, stolen data, or potential fraudulent wire transfers.  Be sure your organization has implemented the latest and greatest security tools and services by having a conversation with your trusted security advisor.

/by