Auto-discovery is not enough: Why  human intelligence is critical to discovery

The IT environments that I & O leaders manage today are becoming increasingly complex and distributed. They’re evolving, making it difficult to completely understand what you have and what is connected to what. Discovery is the method for gathering data to facilitate an understanding of your evolving environment by capturing and recording application, cloud, compute, network, and storage environments—to create or update your CMDB. It’s a critical step in any major IT project.

Most businesses use an auto-discovery tool to conduct discovery. We do, too, for our client engagements. Although there are great tools out there, an auto-discovery tool alone is not enough. Why? The short answer is that discovery requires a high level of people power to collect and analyze the data. We call this complete IT discovery: auto-discovery + human intelligence. 

The limits of auto-discovery

No auto-discovery tool can fully explore a hardware’s total value on the network, including system values, application functions, tasks, and so on. For example, serial-connected devices cannot be discovered through an auto-discovery tool. 

Another example is IP addresses. If you’re using an IP-based auto-discovery tool, it will discover only IP devices and overlook the equipment in your environment that does not use IP. Without this data, you won’t have a complete understanding of your IP addresses and their relationships to other equipment within your site(s). 

Newer auto-discovery tools can track communications between applications by creating data flow maps. But the data alone cannot provide the context needed for discovery such as, for example, if security regulations such as HIPAA or GDPR come into play. 

While a quality auto-discovery tool is essential to discovery, simply deploying a tool is not recommended for large-scale projects. The success of your IT project is directly connected to the thoroughness of your discovery efforts. 

The role of tribal knowledge

Application and server owners know your systems inside and out. Their local tribal knowledge is the best way to completely understand your systems’ relationships. They support and maintain them daily and can provide insight into the daily tasks and overall goals

For example, if discovery is required for a system that has been operating for years but lacks complete documentation due to employee attrition or rotation, critical data will be missed unless you reach out to the program management and the application and technical leads. 

On a recent data center migration project, the client’s management believed that all systems in a disaster recovery (DR) site being decommissioned were backup systems with no operational impact if they were shut down. While scheduling the shutdown of one DR system, we learned from that system’s users that there was in fact a rarely used production system in the DR site. The folks managing the system had moved to other jobs in the company, retired, or left over the last couple of years. We were able to track down the former team and, with their help, migrate the production system with no impact. 

You also need to understand at what level the data is located so it can be correctly gathered. Source out all application, project management, and technical leads that can help fill in the blanks you have in scope. With their help, a better picture of what is being discovered will develop into real data.

This effort will begin to reveal the additional information and people you need to locate for those systems. It allows you to better scan and identify all hardware and software related to your goals and move you closer to data transparency.

RELATED CONTENT  | Infographic

Seven steps to complete IT discovery /

Complete transparency with auto-discovery + human intelligence

When you launch auto-discovery, your tool should scan for devices at each location. This addresses only the IP-reachable side of the devices and software services in use at the time of the scan. Certain systems will be offline at times, so weekly or biweekly scanning will be needed to see those missing IP services so that they can be tracked as well. This type of scanning is critical to observing those devices following special tasks. Skip this step, and it will be difficult to understand their role in the total picture of the site being reviewed.  

Once auto-scanning is in place, the discovery team should upload the data into your CMDB weekly so that the overall data collection can increase over time. Data correction and inspection of incoming data is also important to follow during this phase. This manual process is done by team members on two fronts. First, data coming in for review needs to be correctly imported so that the data is not lost or damaged. Second, although data is delivered in an organized format, it is raw, so each import needs to follow a protocol and be approved as clean before the data can be reviewed. 

Having a data-control lead that matches the data back to each team will assist in understanding the roles/tasks that systems may be performing. It will also assist in understanding how critical the platform is to other systems and to the owners of all IT solutions. 

With auto-discovered data in hand, the team can then classify each system/hardware. We refer to this as tiering the data center assets from Tier 1 to Tier 4, as well as retire, dev, and test classifications. Tier 1 refers to infrastructure/domain controllers and other high priority systems that support most, if not all, of the IT environment. Tier dev or test have almost no real impact to any system. This classification shows the total level of importance of each asset and reveals where each system belongs.

During manual inventory, the team can discover missing elements that may not be tracked, such as fiber channel, serial devices, and other hardware. This should be manually updated into the CMDB and will fill in the last of the missing data on the device side of the effort. Once again, it’s important to ensure that the data is added correctly so that the data-control lead has clean data to review. 

The groundwork for transformation projects

This combined mode of collecting the data, centralizing it in a CMDB, and then reviewing the data will provide a complete picture for each system. In a migration project, it enables the team to build a move-group list by tiering. The team can then decide what the tiers will look like by group and build out documentation for the migration. In a consolidation, this complete data allows the team to make decisions about consolidating, modernizing, or decommissioning servers and other infrastructure. 

Complete IT discovery also supports and sets the foundation for a variety of other projects, including cloud assessments, modernization, infrastructure assessments, and cloud migrations.

The value here is in the manual touch. Auto-discovery tools alone are unable to complete many of the tasks required for complete IT discovery. You need people with the right experience and expertise to validate the data that is auto-discovered and manually seek out data that cannot be auto-discovered. Together, they provide the data transparency needed to understand your environment and complete transformational IT projects.