There is no doubt that as we continue to move further into cloud architecture and infrastructure our environments are evolving daily. This rapid pace of change is failing to keep up with the expectation that your team knows "how it works" and "what it looks like". Keeping network diagrams up to date is often the last thing on most engineers mind, until the time you need them which in some cases is too late.
Manual creation and updating of diagrams are tedious and time-consuming and not that effective with continuous change. These days we have the ability to automate the process of generating and drawing a diagram, getting the information straight from the source of truth.
"Network diagrams should just be something automated"
In the following video by Greg Ferro he explains his opinion about whats wrong with the modern practice of network diagrams - outlining the time and cost of updating and maintaining diagrams:
Video: Two Beer Networking – Whats Wrong With Network Diagrams ?
I think that the time for Network Diagrams is coming to a close.
1. It takes large amounts of time (and thus money) to produce diagrams.
2. Maintaining diagrams is difficult, costly and something that should be automated.
3. Networks are not static today. Overlays, IPsec Tunnels, VMs, virtual appliances. How can a diagram stay up to date with manual changes.
4. A diagram is better than nothing but anything is better than a diagram.
5. Who extracts value from a diagram? Executives, project managers? (Of course, you do but mostly that a by product of writing it down)
"Network diagrams tools will never be particularly successful. We're never going to see a market for network diagrams, and increasingly, that market is actually going away. Why? And I'll talk more about that in a little while, but of course, the answer is ..."
It takes a large amount of time.
"Network diagrams are incredibly hard to do, there are no tools that are optimized for it, yes we all use Visio, I use Omnigraffle for it, but really the time that we've taken to create that stuff is just not a very efficient use of time. It's incredibly expensive. I've spent weeks doing diagrams at highly inflated consultancy rates."
Hava is an automated network diagram tool for cloud providers like AWS and Azure, a product that was borne out of our frustration as IT consultants when it came to begin work on a new project. The answer to the question “so, what does it look like?” was almost always met with an inevitable shrug, so the first few days were spent coming up to speed with the new architecture. Time spent piecing together and mapping out their systems before recommendations be made and the real work commence.
Plans for teams and business for every stage, shape, and size.
Networks are no longer static.
"In reality, network diagrams are pretty awful. They don't really capture any of the information you want. As data centers get more complicated, we're looking at things like VMWare NSX, we're looking at virtual firewalls and virtual appliances. What's the meaning of a network diagram when your virtual appliances are actually dynamic, they actually move around?"
Hava uses automated layouts to display your cloud infrastructure as it is, right now. Different diagrams, with different intents are generated depending on the resources within your environment. Our Infrastructure diagram brings a resource-centric view to your VPCs and Virtual Networks, allowing you to see which instances, databases and load balancers belong to a particular subnet or network. You're able to see connections between resources (e.g. instances belonging to a load balancer backend), drill-down and see resource attributes, security configurations, cost estimates and more. At a glance you're able to see how resources are placed within subnets, balanced across availability zones, and potential misconfiguration or bottlenecks within your environments.
Start thinking about automated diagrams
"What I'm trying to say is, I think the time for network diagrams is probably passing us by, and over the next few years, this idea of manually creating diagrams is probably pretty much done, and we need to start thinking about a post-diagram world where the network configuration is automatically updated in some sort of a platform."
We all need to start somewhere, but in a post-manually-drawn-diagram world, spending time update diagrams for environments that change on a daily basis - those diagrams are only useful for you today, long-term it is becoming an increasingly pointless task.
Not only can Hava help with the initial creation of a diagram, versioning support will allow your team to track and display cloud infrastructure changes over time. Watch how your environments morph and change during their lifecycle. Better yet, make use of our output in a way that suits your organisation and audience. You can view and export our diagrams in many file formats including PNG, PDF, Visio, CSV
The future of network diagrams
Like Greg, we also believe that manually drawing network diagrams are going to be a thing of the past. Connecting directly to the source of truth to automatically generate a visualization of your network will be the way moving forward. Allowing you stay up to date with no effort, and giving your team more time to focus on the things that actually matter.
AWS NETWORK DIAGRAM GENERATOR
Sitting down with a drawing package like Visio or Lucidchart and your AWS Icon template pack is a daunting prospect. It's even more challenging if your AWS environment contains more than a handful of VPC's, EC2 instances or a load balancer or two.
Not only do you need to manually draw out your VPC's and resources which can take hours or even days, once you have drawn the diagrams, you then need to keep them up to date if they are going to be of any practical use. As we all know, the minute you think you have your diagrams up to date, something changes or autoscales and your diagrams are out of date. That's where using an automated AWS diagram generator comes into play.
AWS NETWORK ARCHITECTURE DIAGRAMS
There are a number of Hava diagrams that can prove useful to your engineering and DevOps team. These include an AWS Infrastructure diagram like this:
The above diagram example logically lays out all the resources discovered when an AWS account is connected to Hava's AWS Network Diagram Generator. The main VPC denoted by the green border, is surrounded by associated resources like internet and VPN gateways, S3 Buckets, VPC endpoints, VPC peering connections, Elastic Beanstalk environments and so forth.
Within the VPC, the configured availability zones (AZs) are set out in columns that contain the individual subnets set up in those AZs. All the resource instances contained in each subnet are visualized, as are any load balancers routing traffic to the various subnets.
When you have a diagram automatically created from configuration data, now you can interrogate of each of the resources to see the settings and associations related to it. By clicking on any of the individual instance icons, the attribute pane on the right then displays all the known details related to the resource. Something that simply isn't possible with manually created drag-and-drop diagrams.
Automating the diagram build using an AWS Network Diagram Generator can also reveal resources you are paying for, but may not be aware of. Long forgotten database instances and sometimes entire dev or test environments are regularly discovered by this automation process. One of our clients discovered a database instance that was unused and costing close to $3k per month, and had been for over 3 years!
CUSTOM AWS NETWORK TOPOLOGY DIAGRAM
There is always somewhat of a compromise when it comes to automatically generated documentation. The way that Hava discovers and builds diagrams is based at a VPC level. If more than one VPC is detected in your AWS cloud account, then one diagram set per VPC is created.
You may however want to combine two or more VPCs onto a single diagram, for instance if they both relate to the same project or application. This can be achieved using the custom query search function built into hava.io.
AWS NETWORK TOPOLOGY DIAGRAM GENERATOR
This custom query would create a new 'on-the-fly' custom diagram containing the two nominated AWS VPCs specified in the search. You would then have the ability to save this diagram so it is retained in your dashboard until you choose to delete it. As with all other diagrams generated by Hava, your custom AWS infrastructure diagram would keep itself up to date and retain a version history every time a resource change is detected.
AWS RUNNING COMPONENTS DIAGRAM
Typically it is just not practical to include every single component onto an AWS network diagram. Take for instance network interfaces, volumes or WAF Rules. In a large network with hundreds or maybe thousands of these non essential components, trying to map out every single component would most likely make the diagram unreadable, or certainly too rammed full of unimportant resources to easily recognise the core components.
However, that's not to say you don't need to know about them.
Hava solves this issue by providing detailed components list that lists out every single resource detected.
This comprehensive list of resources can also be sorted and exported along with estimated monthly costs so you can see at a glance what resources are costing you the most money. This detailed view is also interactive. Selecting a resource on the diagram will reveal all the known settings and associations that resource has.
AWS SECURITY GROUP DIAGRAMS
Another benefit of automating the discovery of your AWS environment diagrams with Hava is the ability to capture and visualize security group, traffic and open port details.
AWS SECURITY ARCHITECTURE DIAGRAM
With the Hava security group diagram you can view all of your configured security groups with the open ports overlaid to enable an instant visual snapshot of the traffic flow, ingress and egress points. The security group diagram being interactive ensures you can select a particular security group on the diagram and view important information relating to the group, like the connected resources, ingress and egress ports, IP addresses and associated tags.
AWS ARCHITECTURE DIAGRAM VERSION COMPARISON
Hava continuously scans your AWS architecture and when changes are detected a new diagram set is automatically generated. The superseded diagrams are not discarded or overwritten. Instead they are moved into version history. Still fully interactive.
What this means is you can view your cloud architecture at any point in time and also leverage Hava's revision comparison (Diff Diagrams) to quickly identify what has been added or removed between the two diagram dates.
So you can easily identify all the changes made since your last compliance audit, or see what changed yesterday that is causing unexpected network or application errors.
AWS ARCHITECTURE MONITORING
While diff diagrams are super helpful in diagnosing changes after the fact, you may want to keep on top of changes as they happen.
Hava's architecture monitoring alerts will let you know the minute a change is detected. You simply nominate the environment you wish to monitor and add a group of recipients to receive the alerts. When a change is detected like the addition or removal of a resource, Hava will send each recipient a diff diagram showing the changes.
Now you and your security team can be across every change as it happens so you can assess and take action if required.
ENVIRONMENT DIAGRAM NOTES
For every architecture diagram generated you have the ability to add text comments. This serves as a rolling dialogue your team can contribute to that may better explain elements of the diagram or bigger picture concepts related to the diagram.
Notes are accessed from the accordion menu within the attribute pane.
New notes are added to the top of the list so they are stored in chronological order.
AUTOMATED AWS DIAGRAM UPDATES
Finally, your documentation is only useful if it is accurate. It is quite possible to spend weeks manually constructing network topology documentation only to have it rendered useless by a minor change to your network configuration.
In the middle of an outage or network incident that has taken down your application, being able to quickly establish what should be running is crucial.
Built into Hava is an auto-sync function that polls connected data sources and automatically updates diagram sets when changes are detected. This means your diagrams are always current and up to date. The superseded diagram sets aren't discarded however. They are placed into a version history that enables you to pull up older diagrams in the same fully interactive format.
This allows you to quickly and visually identify changes to your cloud network topology, so you can easily compare a previously working set of network resources with the current configuration to see what's missing .
It also allows you to demonstrate the status of your network at any point in history. This can be invaluable during a pci compliance audit or insurance claim should your network design ever be called into question.
EXPORTING GENERATED AWS DIAGRAMS
Native AWS architecture diagrams created by Hava are the nearest we've seen to the examples and recommendations provided by AWS. These are great to view and interrogate via the interactive dashboard, however sometimes you'll need to pull a set of diagrams for audit purposes or for management or sales presentations.
Hava's built-in export function allows you to do this in a number of formats.
CSV, VSDX, JSON, PDF and PNG
EDITING AWS NETWORK DIAGRAMS
Should you want to manipulate or embellish your Hava generated diagrams, exporting to VSDX format and using Visio, draw.io or any VSDX compatible drawing package will allow you to edit your diagrams as required.
Should you not have access to Visio but would like to try this out, try opening one of your exported VSDX files in draw.io
Getting Hava to do the bulk of the heavy lifting by initially generating accurate diagrams based on what is actually configured and running in your AWS environment enables you to access a base diagram ready to edit which will save you hours or possibly days preparing management reports or upgrade plans.
In Conclusion.
There aren't too many engineers that would dispute the benefits of perpetually accurate network topology documentation. Hava provides just this with:
- Auto generation of documentation for your AWS environments (as well as GCP & Azure)
- Keeps the diagram sets updated
- Retains a full set of diagrams in version history every time a resource changes
- Provides a comprehensive API to allow IaC build pipeline integration
- Provides a unique security view detailing security groups with visualized traffic ingress/egress
- Coming soon true 3d and separate compliance reporting
- Available as SaaS or fully self-hosted
You can find out more about Hava here:
