LIVE: Hello! Imagine a thing with human faces, what a treat, I get to stand up, not worry about being on mute, use my clicker and everything! This is exciting! RECORDING: Hello!

Ok, elephant in the room, policy is a dull thing, its kinda hard to make sexy, but I'm going to try and get your attention, so bare with me.

So to set the scene I'm in the lift (yes, American friends we really do call them lifts) four people walk in. I think to myself, Chris! this is your moment, now or never. as the door closes, I position myself in front of the door, a captive audience, they're mine, I've got them. I hear the doors seal shut behind me, I take a breath.

I look at the first person on my left, she's in a suit, she looks really important, I gesture to her C...

she looks back to me as if to say yes, go on, eye she nods, Oh. ok perfect the CIO the policy maker, the one who's neck is on the block, what are the chances of finding you in my imaginary lift today? I ask her, what keeps you up at night?

She tells me I don't know what teams are really doing what, the volume of risk and what I should show more interest in.

Setting and changing policy is slow and hard to communicate

People just go off and do their own thing, they think they know better, and to be honest often they do, but I'm left playing catchup with the risk they've signed me up to

Ok, I say. trying not to sound like a patronizing snake oil salesman I can help.

I turn my attention to the second person, in a suit, look less important. I make a guess, (lets face it, this is my imagination it'd be weird if I was wrong). Product manager I say, they nod. ah the whip cracker I say. What's important to you?

Managing risk, mostly opportunity risk, the fear of missing out. so getting features out the door, avoiding getting bogged down with

(they glance to the CIO) bureaucracy that is designed to slow me down.

Awesome I say, this is your lucky day.

I turn to the next person, dressed in overalls, I am in a trendy part of town, they could be the CTO, before I ask they sense me staring at them.

Cleaner they say, errr ok how did you get in my imagination, let me come back to you.

My attention goes to the last person, hoodie, headphones round their neck. Ah my stereotypical developer, yes I know you well.

What code do you write I ask (it doesn't matter), python, cool have you got everything updated to work with, I pause, Python 3? they say, yeah, python 3, that must be hard I add. They don't know it yet but I've won their trust which is important. Nearly they say. cool. Whats important to you?

Staying on top of patching dependencies, so we can react to the next fire.

Knowing what rules exist, what ones I can bend, what I can break, and what might cause me to loose my job.

Writing consistent good quality code, avoiding technical debt, the rest of my team being able to cohesively work as one.

Use any tools to help you with that I ask? Yeah, linters, code quality and test coverage tools, the usual.

Great I say, I write code too, lets be friends, I hand them a printed QR code here's my public gpg key, so you know to trust me what I say.

I return my focus to the cleaner, I've got it, How do you get told what to do, and when it changes.

We get a memo, or something stuck to the notice board, last week we got a memo saying that all the meeting room whiteboards need to be cleaned every night. Interesting I say, how does that work out, well its up to us to then maintain the todo list, so we can on board new people.

Go wrong at all? yeah, sometimes if when we compile our operational manual we miss a memo, or don't apply them in sequence we get things wrong,

they glance apologetically to the product manager, like when we hadn't updated the guide that the meeting room on the 3rd floor was being used as a dedicated war room, and we wiped their boards down.

I look to the dev, sound familiar I ask, they nod. Turns out we're not all special snowflakes hey! Ah all is not lost, I knew there was a reason I imagined you here.

The lift is slowing, I feel it coming to its destination Great, I have got the silver bullet for you too.

The CIO looks to me ready to buy whatever it is I'm selling, they ask me as the doors open who are you, what team are you in?

Ok so if any of that sounds familiar, and you relate to my imaginary friends then I've got the answers for you.

What if I said:

You could update policy easily, even releasing several version updates, not just in a year, a month

what about ten updates in a single day, and seamlessly communicate that to people that need to consume it without derailing them?

You could have visibility on compliance using tools you already use?

That policy could be readily consumable, easy to parse, demonstrate compliance, make sense and not be bureaucratic to change when it needs to be and not get in the way?

That same policy could be treated as a dependency, and operate like a linter, so you can run compliance checks locally, in CI and guard production ultimately

That multiple versions of policy like a dependency are supported, so emergencies like you must update now because theres now known vulnerabilities type updates are a business as usual activity to communicate.

Interesting, ok, hang around.

LIVE: HOW AM I DOING FOR TIME? .... GREAT, THANKS, I THINK I'VE GOT THIS, YOU CAN UNLOCK THE DOOR NOW. RECORDING: Lets crack on

Now I've hopefully got your attention, its time to introduce, and start explaining myself. Hi, My name is Chris Nesbitt-Smith, I'm currently an instructor for Learnk8s and ControlPlane, consultant to the CDDO and tinkerer of open source. I've spent a fair chunk of my professional career now working in UK Government and large organizations where problems like these are rife. I've been promised we'll have time for questions and heckles at the end, even if I run way over, if not come find me at the bar afterwards.

LIVE: Given I've got the luxury of an audience, and most of you have clothes on for a change. By show of hands who's with my CIO and set, written, or applied policy before? RECORDING: While this is not live or in person where I could ask you to raise your hands, we can still try some audience participation, so if you could leave a comment of "policy maker" if you're with my CIO and have set, written, or applied policy before.

LIVE: Ok, thanks, you can put your hands down. now how many have sought exemption or consciously bent, broken, circumvented, ignored, bypassed, whatever a policy with at least good intentions? RECORDING: Ok, next round, if you have ever sought exemption or consciously bent, broken, circumvented, ignored, bypassed, whatever a policy with at least good intentions leave a comment of "policy breaker"

HA, you fell for it.

We've got all your names and employers details, so put your phones down, lend me your ears, the stakes just got raised.

Where do I see policy as code going wrong?

Before we dig into that, what do I mean by policy

It usually comes in one of two forms

Security enforcing, like data at rest being encrypted

Consistency enforcing such as code style, tabs being two or four space indentation maybe

Or maybe you can think of some others, but in any case its hopefully intended to mitigate a risk of some sort

However with the best of intentions these are often emotionally led rather than being grounded in a proportionate control which is the open door to case-by-case exemptions being required when you come against a situation you weren't anticipating

This is not unlike how the laws of the land are created, with case law making for a complex to navigate rulebook, and harder still to measure compliance

It often looks like the thin bit of a wedge, where the precedent which may have been an uncomfortable pill to swallow the first time round, becomes dangerous with others looking to expand its scope.

which can lead us to sometimes wonder if the cure is actually worse than the disease

but thats not how we (at least typically) develop software, so why does this have to be so hard? there must be a better answer

we've codified everything else so isn't this the answer, well yes in part but my point of this talk is we do it wrong

Maybe some of you are screaming your favorite product name at me in your heads as the solution, and you're not wholly wrong

Devil in the detail

throwing some curly braces or yaml at something doesn't inherently fix things.

If it's a security control its often tempting to keep the policy a secret, exposing it could maybe be used against you by an adversary

however that does not support us shifting left at all, it results in devs effectively reverse engineering what the policy is by finding out when we smash our heads against it

it doesn't take much imagination to see that in the scenario of an application deploy midway through finding one resource is non-compliant and rejected would leave the overall deploy in an inconsistent halfway state likely resulting in downtime

Which begs the question of was the policy better than the downtime

Especially if it leads your engineers who are hopefully all plenty smart finding 'inventive' shall we say ways around the

computer says no response they got.

this is further exasperated when updates to the policy are desired, maybe you get a pen test or something goes wrong so you form that case law and need to apply a new policy, maybe all s3 buckets now need to be encrypted, a change that could be considered a 'breaking change'

sure you might say that you provide warnings, on at least the less important issues or new emerging policy, which is great

So long as someone sees them

But if you've adopted gitops or at least CI/CD is anyone seeing those warnings? Who studies the results of a successful build log every time?

anyone, every time?

Well if you are, I'd politely suggest you're probably missing the point of CI/CD, you should be able to trust your job status

Ok, well I'm not here today to just throw stones

Remember my implied promises to my four imaginary friends of what the promised land might look like?

well theres nothing new under the sun, we've already unwittingly solved these problems elsewhere, we just need to be reminded and join the dots

well the first is something if you're doing policy as code, you're probably already doing, put it in version control. the thing you might not be doing though is then making that visible

so at least inner source this, by which i mean allow anyone within your walled garden (employees, suppliers etc) to see the policy, I'm not saying give all your threat monitoring rules and intel away, you can probably keep those to yourself, but I'd argue visible policy and the gaps therein is often better than downtime, reverse engineered workarounds, and opaque legacied exemption spaghetti soup.

If you're brave you might even open source it, you'll find it unlocks the ability to work well with prospective suppliers without NDAs and what not, and widely distributed secrets are expensive to maintain, difficult to handle and often only stay secret for so long after all

Ok, we're off to a good start, our policy is visible now to those that need to see it

many of you are no doubt used to semantic versioning, but a quick recap

The first segment is to indicate breaking, perhaps conflicting changes, in the context of policy, lets say it's requiring resources to have a department label, maybe that'll help with some internal cross charging, who knows, I'm not judging

An increment to that might look like requiring that to be from a predetermined list rather than be freetext

The second segment is to indicate minor changes, that shouldn't break anyone

An increment to that might look like correcting a spelling mistake one of the department names

The third segment is to indicate patch changes, these should be a no brainer to keep to to date with

An increment to that might look like adding a department to the available options

Ok so our policy is visible in a repository now, its versioned so we can easily communicate the policy, we can tack on release notes, and expectations are managed by semantic versioning

in software we're used to handling dependencies, so what if your policy was just another dependency

you might unwittingly already be doing this if for example you have eslint as a dependency in your javascript package perhaps?

Ok so our policy is visible in a repository now, its versioned so we can easily communicate the policy, we can tack on release notes, and expectations are managed by semantic versioning, you know, like software!

ok, i know testing is a dirty word, but in order to make this an asset everyone can depend on, and also provide known good examples, tests are essential to give

everyone confidence in the stability and surface potential side effects before they hurt everyone involved.

consumers of this policy need to be able to test themselves against the policy locally and in CI/CD

thus shortening the feedback loop better informing

so as a bonus, we should find our consumers able to rely on the artifact we're sharing with them

we're well and truly on the home stretch, its a dependency so updating it should be no different to any other

we can even use some magic like github's dependabot or mend's renovate to do that for us, think automatic pull requests, tests, even auto merging if you like

ok, to check you're awake still, can anyone tell me a recent event that caused people to want to know what version of a certain logging java doohickey you were potentially running literally everywhere in the estate?

yup, as you know all presentations this year are contractually required to reference log4j

even when its almost entirely out of context

and include some memes. in just a few short months I can remove these and hopefully

just point broadly at a list of scary looking CVEs in order to command your behavior through fear https://www.cvedetails.com/vulnerability-list/cvssscoremin-9/cvssscoremax-10/vulnerabilities.html

What I'm getting at here though is, the situational awareness piece around software supply chain is something your organization is hopefully already

thinking about if not already addressing, so if our policy is a dependency this at least not a new problem. Software Bill of Materials for the win right?

Which can allow us to measure the compliance across the estate

I've just covered a lot of ground, and hopefully sounded convincing, and not just a fictional utopia painted in powerpoint

its time to look at how you might be *able* to do this and

I know you really came here wanting to see a million words on a slide not just an emoji or two

so we've reached the point where I show you some code, hooray!

to maintain scope, I'm going to limit this to talking about two things, to prove its not just one tech, or tool.

I've arbitrarily picked terraform and Kubernetes

but I could have picked anything

naturally I'll need some tools to go along with this, I'm too lazy to invent anything here

so likewise I'm going to pick two tools, but again I could use any, some, or even all, probably. Checkov will do my terraform, kyverno will to my kubernetes.

If you want to browse along with me, I've created a example git hub organization here, I'm not expecting you to read or grok the code on screen, so don't worry about it too much, its just to prove its a real thing.

The policy is stored here

so heres where my policy starts at v1.0.0 I've got policy that requires a department label on all resources, so long as its set, doesn't matter what it is

I've written tests for this, note how the passing test cases are usable as a great example of what good and bad looks like

we've pushed a tag in git, we've added release notes, I can sign it to provide further assurance if my heart so desires.

it does (obviously) moving on

version 2.0.0 looks similar, only now that department field has to be one of a predetermined list, like before, tests exist, release notes are written, tags are signed

2.1.0 is where we notice and correct a spelling mistake of one of the options in that list of departments

2.1.1 and I've now added a new department to the list.

app1 and infra1 depend on version 1.0.0 of the policy, it is not compliant with version 2.0.0 or beyond, but how do I know that?

I've configured renovate to automatically make a pull request

when theres a new version of the policy, so its super obvious if I can update my dependency, and I can see

clear feedback about where and why I'm not compliant

I can also see all the pull requests over the org, so I can measure the compliance of my policy https://github.com/pulls?q=is%3Aopen+is%3Apr+archived%3Afalse+user%3Apolicy-as-versioned-code

moving on from that app2 and infra2 depend on version 2.0.0 of the policy

however we could merge the open pull request all the way up to 2.1.1

finally app3 and infra3 are dependent on 2.1.1 they get a gold star from the CIO

there is a small touch of magic, and its not pretty

I've written some bash

don't judge me, even though I've probably, definitely, written worse

what this does is allows me from my dev laptop or in CI to evaluate my code against the version of policy, ideally this might be less cumbersome, but it is what it is for now, pull requests are welcome!

and the last piece to the puzzle is managing the lifecycle of the policies, and allowing multiple versions of policy to be accepted and evaluated within a single runtime

I've cheated a bit here, kube gives you admission controllers, its not so easy to get the same policy evaluation in a cloud, they've got their their own policy code, I've not figured how to be able to evaluate that policy locally, again pull requests, contribution, collaboration are all very welcome

You may have noticed the way the policy is designed and distributed lends it self well to co-exist in a Kubernetes cluster

which brings us to the cluster1 which describes a cluster that accepts all the versions we've described so far

likewise cluster2, only accepts 2.0.0 and greater

we automate using KiND for CI to deploy the apps

and there we have it a full org all done, all compliant, policy all versioned, CIO all aware of whats going on. So this is great,

but just one more thing, wouldn't it be awesome if the the policy carried a story of why it exists

after all if your agile team is even half effective it will reject anything it perceives as friction if it doesn't see value in it

it could allow our devs to know why they're compliant, and if they want to do something outside what the policy permits, they don't need any sort of exemption granted per-say, they can have a well reasoned and informed debate with rationale behind a pull request to the policy

imagine, if you will, this going through a stage of versions, with risks that inform the mitigations manifested as policy maintained as one. so when the risk landscape changes, your policies can move with it

when some new privacy regulation comes out, or your latest marketing strategy pays off and you acquire more data for example, even if your policy was perfect at one time, the risks and the appetite stand still for no one

we can liken this to over provisioning, that we might be familiar with elsewhere, where lead times are long, change is hard, and there is a significant pressure in nailing it first time, which can lead to hedging bets against what some future state might be, rather than proportionate mitigation to risks that are more tangibly real in the now

thats where the real culture change is needed, and the execution of that is likely a long series of talks in itself

So now this is really over to you, honestly the best thing you could do right now is tell me its madness, already done, irrelevant or otherwise unachievable; something my esteemed echo chamber of peers has yet to do.

So beyond making pull requests and developing the theory more, I'd really like to start building a case study with a willing real organization, and allow me to swap out my imaginary friends for real ones

But the most important thing I want you to remember from our time together is that. And feel free to say it out loud with me

Purposeless

policy is

potentially

practically

pointless

policy.

I've been practicing saying that far too many times.

I've been Chris Nesbitt-Smith, thanks for your time, you're now free to leave, I'll destroy the evidence of your guilt admissions earlier - no really I will Like subscribe whatever the kids do these days on LinkedIn, Github whatever and you can be assured there'll be no spam or much content at all since I'm awful at self promotion especially on social media. cns.me just points at my LinkedIn. talks.cns.me contains this and other talks, they're all open source.

LIVE: Questions are very welcome on this or anything else, I'll hold the stage as long as I'm allowed, or find me afterwards, I'm pretty thirsty so I'll be over there. RECORDING: Questions are very welcome on this or anything else, I'll be hanging out in the comments section, or ping me a message on LinkedIn.