So you just landed a new cloud customer and it's your job to work out how their existing cloud architecture has been cobbled together over the years.
Maybe they are primarily an AWS shop, or prefer Azure or are cool kids and have gone all in on Google Cloud. Or maybe they are uber cool and have adopted a hybrid cloud approach. Either way you have some long tedious hours or days ahead of you trying to unravel exactly how these multiple vendor accounts hang to together. By days, we obviously mean weeks.
Maybe you have been tasked with increasing application performance or streamlining the many disconnected workflows to provide a more stable, more available application that continues to perform under increasing traffic load. Or perhaps it's your job to ensure the systems are robust and have enough redundancy built in to survive a regional cloud vendor outage.
Either way, from our time providing expert cloud consulting services back in the day, the very first task (after finding the coffee machine) was to always gather all the production, development and staging/testing cloud accounts and map out all the environments in diagram form.
This meant combing through all the client's cloud consoles and building a picture of all the virtual networks that were configured including the test/dev VPC's that were running but not necessarily in use. This could take days or sometimes weeks. Discovering running resources, placing them in the correct VPC diagram, working out the security profiles, how traffic or other services accessed the resource, how peering connections allowed communication between VPCs, establishing which VPCs were related to production environments and which were testing, staging or backup snapshots. It could take weeks to properly establish exactly what was running where and how important each resource was. Weeks of mind numbing tedious grunt work trawling through consoles, building spreadsheets for each VPC and finally busting out a drag a drop drawing package to try and visually represent what was going on.
There were obviously far more productive things we could be doing for the clients like actually fixing the network, redesigning a faster, more secure, more available environment. Don't get me wrong, the time spent diagramming the networks was always great revenue, but if you're anything like us, you'd rather be doing some impactful work instead of drawing diagrams.
So the solution to dramatically reduce the time it takes to document a client's network topology is to automate the process, or more specifically with Hava, the system we developed to automatically discover and document client networks as part of the onboarding process of clients we took on that had workloads on AWS, Azure and Google Cloud.
From a consulting perspective the ability to automatically uncover what a client has running in the cloud, or more specifically what is ACTUALLY running as opposed to what they assumed was running is a valuable starting point. Uncovering unused resources, test environments and glaring security holes is an easy win and not something you would necessarily go looking for if you were creating diagrams manually.
By connecting cloud accounts to Hava, in a matter of minutes you will have a diagram generated for every VPC or virtual network discovered, which details all the services and resources deployed in that account.
It's not uncommon to see unexpected resources sitting in an orphan VPC, maybe as a result of a quick test someone did one day and forgot to delete. It's also common to come across open ports on security view diagrams that shouldn't be there. Maybe they were opened by mistake, or temporarily opened for developer access, but forgotten about and now they provide easy access to the client's production environment resources for those inclined to poke around. These are all surfaced when you auto generate network topology diagrams using Hava.
The massive time saving is only one reason that Hava is the best diagram tool for cloud consultants. Another reason is that once created, your diagrams keep themselves up to date. Automatically, hands free. This means at any time you need a diagram, it's ready and waiting for you and reflects the current state of the network.
When you are redesigning, tweaking or generally modifying a client's network or have multiple teams working on projects that use the infrastructure you are working on, an audit trail of what changes day to day is also invaluable. When Hava detects a change and generates a new diagram set, the superseded diagrams aren't deleted, but are moved to version history. This means that if you are trying to track down a problem, the offending change hasn't simply been overwritten in between your logins. The change is documented and visible in versioning.
So what do you get when you add Hava to your cloud consultant toolbox?
AWS NETWORK TOPOLOGY DIAGRAMS
There are a number of cloud visualization diagrams that can prove useful to your cloud consulting work day. These include an AWS Infrastructure diagram like this:
The diagram shown above logically lays out all the resources discovered when an AWS account is connected to Hava's cloud visualization tool. The main VPC is depicted by the green outside border and is surrounded by associated resources like internet and VPN gateways, S3 Buckets, VPC endpoints, VPC peering connections, Elastic Beanstalk environments and so forth.
Inside the VPC, your configured availability zones are set out in columns. These columns contain the individual subnets set up in those AZs. All the resources contained in each subnet are visualized as are any load balancers that are routing traffic to the various subnets.
Having your diagrams automatically created from your AWS console configuration data enables interrogation of each of the resources to see the settings and associations related to them. By selecting any of the individually visualised resources, the attribute pane to the right hand side of the diagram displays all the known metadata related to the resource. This is one aspect of Hava's cloud visualization diagrams that isn't possible with manually created drag-and-drop diagram makers.
Automating the diagram build using Hava also reveals resources you may not be aware of. Long forgotten database instances and sometimes entire development, staging or test environments are regularly discovered by this automated cloud visualization process.
GCP CLOUD ARCHITECTURE DIAGRAMS
Hava's cloud visualization diagram tool will connect to your client GCP accounts and visualize your GCP networks, zones and subnets.
AZURE CLOUD ARCHITECTURE DIAGRAMS
In addition to visualizing your AWS and GCP cloud environments, you can also import your Azure environments into Hava. Hava will create diagrams for each resource group discovered in Azure. The resource group will detail any virtual networks found in the resource group which in turn can contain multiple subnets and other resources like virtual machines, load balancers, peering connections and storage accounts.
CUSTOM CLOUD VISUALIZATION
There is always a compromise when it comes to automatically generated documentation. The way that Hava discovers and generates network topology diagrams is based at a VPC or resource group level. If more than one VPC is detected in your Amazon cloud account for instance, then one diagram set per VPC is created.
You may however want to combine two or more VPCs onto a single diagram, or just diagram a subset of your resources. This can be achieved using the custom query function built into Hava.
CUSTOM CLOUD Diagram GENERATOR
This custom query would create a new custom diagram containing the two nominated AWS VPCs specified. You would then have the ability to save this diagram so it is always present in your dashboard until you choose to delete it. As with all other diagrams, your custom infrastructure diagram would keep itself up to date and retain a version history every time a resource change is detected. The filter and deep search allows you to quickly find resources across hundreds of cloud accounts using resource names, tags, IP addresses and a range of other search criteria.
CLOUD RESOURCES LIST
In reality it is just not practical to include every single component detected in your cloud config onto a network topology diagram. Take for instance network interfaces, volumes or WAF rules. In a large network with hundreds or maybe thousands of these second tier components, trying to visualize every single component would make the diagram almost unreadable, or certainly too busy to easily recognise the core components.
Hava's cloud visualization tool solves this issue by providing detailed components list that details every single resource detected. This is known as the "List View"
This is a comprehensive list of resources that can be sorted and exported along with estimated monthly costs so you can see at a glance what resources are costing you the most money. This detailed list view is also interactive. Selecting a resource on the diagram will reveal all the known settings and associations that resource has.
AWS SECURITY GROUP VISUALIZATION
Another major benefit of using Hava to automate the discovery of your AWS environment diagrams is the ability to capture and visualize security group information. While only available for AWS currently, security views for Azure and GCP are on our development roadmap.
AWS SECURITY ARCHITECTURE DIAGRAM
With an AWS security group diagram you can view all of your configured security groups. All your open ports are overlaid on the visualization to enable an instant understanding of the traffic flow, ingress and egress points. The security view diagram is also interactive which ensures you can select any security group on the diagram and important information relating to the security group, like the connected resources, ingress and egress IP addresses and associated tags are revealed without having to exit out to your AWS console to investigate. Hava also provides an Azure security diagram.
ARCHITECTURE DIAGRAM VERSION COMPARISON
Hava continuously scans your cloud architecture and when changes are detected a new diagram set is automatically generated. The superseded diagrams are not discarded or overwritten. Instead they are moved into version history. Still fully interactive.
What this means is you can view your cloud architecture at any point in time and also leverage Hava's revision comparison (Diff Diagrams) to quickly identify what has been added or removed between the two diagram dates.
So you can easily identify all the changes made since your last compliance audit, or see what changed yesterday that is causing unexpected network or application errors.
Cloud ARCHITECTURE MONITORING
While diff diagrams are super helpful in diagnosing changes after the fact, you may want to keep on top of changes as they happen.
Hava's architecture monitoring alerts will let you know the minute a change is detected. You simply nominate the environment you wish to monitor and add a group of recipients to receive the alerts. When a change is detected like the addition or removal of a resource, Hava will send each recipient a diff diagram showing the changes.
Now you and your security team can be across every change as it happens so you can assess and take action if required.
ENVIRONMENT DIAGRAM NOTES
For every architecture diagram generated you have the ability to add text comments. This serves as a rolling dialogue your team can contribute to that may better explain elements of the diagram or bigger picture concepts related to the diagram.
Notes are accessed from the accordion menu within the attribute pane.
New notes are added to the top of the list so they are stored in chronological order.
AUTOMATED CLOUD VISUALIZATION UPDATES
Finally, your documentation is only useful if it is up to date. It is quite possible to spend weeks manually constructing network topology diagrams only to have them rendered useless by a minor change to your network configuration. Given the nature of services provided by the major cloud vendors like autoscaling that provision or remove resources in response to traffic demand, the chances are your cloud infrastructure is changing more often than you would first imagine.
Built into Hava is an automatic synchronisation function that polls connected cloud account data sources and automatically updates cloud visualization diagram sets when changes are detected. This means your diagrams are always current and up to date. The superseded diagram sets aren't deleted or overwritten however. They are placed into Hava's version history. This audit trail of changes stores superseded diagrams which enables you to pull up and investigate older diagrams in the same fully interactive format as the live current diagram set.
This allows you to quickly identify changes to your cloud network topology either visually by placing diagrams side-by-side or by exporting them for programmatic comparison.
Versioning also allows you to demonstrate the status of your network to key stakeholders at any point in history. This can be invaluable during a PCI compliance audit, insurance or security audit should your network design ever be called into question.
EXPORTING CLOUD Diagrams
The native cloud architecture diagrams created by Hava are the nearest we've seen to the examples and recommendations provided by the major cloud platforms. These are great to view and interrogate via the Hava application dashboard, however sometimes you might need to pull a set of diagrams for audit purposes or to incorporate into management or sales presentations.
The built-in diagram export function allows you to do this by providing these five export options
CSV, VSDX, JSON, PDF and PNG
When viewing any of your Hava diagrams, you can select the export function, tell Hava what format you would like and Hava will generate the export file for download.
EDITING YOUR CLOUD VISUALIZATION DIAGRAMS
Should you want to manipulate or embellish your auto generated Hava diagrams, exporting to VSDX format and using Visio, draw.io or any VSDX compatible drawing package will allow you to edit your diagrams as required.
Should you not have access to Visio but would like to try this out, try opening one of your exported VSDX files in draw.io (diagrams.net)
Getting Hava to do the bulk of the heavy lifting by initially generating accurate diagrams based on what is actually configured and running in your cloud environments enables you to access a base diagram ready to edit which will save you hours or possibly days preparing management reports.
In Conclusion.
There aren't too many cloud consultants that would dispute the benefits of perpetually accurate network documentation. Hava provides just this with:
- Auto generation of documentation for your AWS, GCP & Azure networks
- Keeps the diagram sets updated
- Retains a full set of diagrams in version history every time a resource changes
- Provides a comprehensive API to allow IaC build pipeline integration
- Provides a unique AWS security view detailing security groups with visualized traffic ingress/egress
- True 3d and separate compliance reporting
- Available as SaaS or fully self-hosted
You can try out Hava for 14 days for free, learn more using the button below:
