If you work in the Government acquisition world, this podcast is for you. (not just for Contracting Officers!)
Kevin and Paul talk government source selection evaluation criteria. Listen and learn how using threshold requirements and objective requirements can give both government and industry the opportunity to reach beyond “good enough” as well as the downsides of incorporating this flexibility into a solicitation.
_________
If you enjoy this podcast, check out Skyway Acquisition at skywaymember.com to begin your relationship with our team of former contracting officers. The Skyway Community is the essential resource for context in the government market.
Give us a call at 877-884-5280 or email us at connect@skywayacquisition.com
LinkedIn CO Podcast Group: https://www.linkedin.com/groups/6796606/
Facebook Government Contracting Network Group: https://www.facebook.com/groups/GovernmentContractingNetwork
___________
Kevin Jans and Paul Schauer created the Contracting Officer Podcast to help government and industry acquisition professionals understand more about how the other side thinks. As former government Contracting Officers who have also worked on the industry side, Kevin and Paul share their perspectives in support of the podcast’s mission: Make government contracts better, one contract at a time.
___________
Transcript Below
Paul Schauer 0:08
Welcome to the contracting officer podcast. It’s not just for contracting officers. If you’re anywhere in the government acquisition world, this podcast is for you. Today, we’re talking government source selection evaluation criteria. And this podcast is brought to you by Skyway acquisition, visit Skywayacq.com. To learn more. Okay, let’s get started. To compete contracts effectively, and and buy what what they need to solve their problems, the government team must have a clearly defined requirement.
Kevin Jans 0:46
In other words, we need to know what we’re looking to buy, and specifically, what we’re running a competition for. The government needs to have a clearly defined requirement to be able to compete between offers. But here’s the catch. That’s not easy, especially when the government team doesn’t know exactly what industry is capable of. I mean, what if we write a requirement too narrowly? Or we set the the spec below the industry standard? Because we don’t know right? Or what if we misunderstood the technical capability that industry has, and we end up asking for something that’s too easy or outdated, raise my hand because I’ve done that. And then we end up awarding a contract. It’s it’s out of date, once it’s awarded, which we hear about that a lot and technology. So
Paul Schauer 1:31
One way to mitigate some of these challenges with these risks is to use thresholds and objectives in the evaluation criteria, a threshold is the minimum that you want, or that you’ll accept. The objective is the goal that you’d like to meet. This is this is my best case, I’d really love if it did this. And before we get into all that, let’s stop and say thanks.
Kevin Jans 1:56
Thanks. This week goes to Stephen Crosby. Steven is an Air Force Contracting Officer. And he’s a program manager on the F 35 modernization program. He’s in the DC area. We found Steven through the contracting officer podcast group on LinkedIn. And I want to thank Stephen for his feedback on our podcast is it’s so great to hear that the program manager and Air Force program manager, not just contracting people found our podcasts helpful. And then he took the time to let us know. So it makes me smile to know that all the open engagement across contracting program management, etc, we really are fixing government contracting, one relationship at a time.
Paul Schauer 2:35
Thanks, Steven, we appreciate your perspective. Back to our topic of the day, what is the threshold and objective? Or even more? What do you do about them? I lay said the threshold is the minimum acceptable, acceptable value for whatever parameter you’re trying to specify? The objective value is what you really want. Or if you’re in the government, what the customer really wants. If you’re the if you’re the acquisition team, you may not care what you want, the customer is telling you what they want. They may realize that their objective isn’t possible isn’t feasible at current state of the art with current technical capabilities or with my budget, but I’m not sure so give me a floor, which is a threshold and a ceiling. This is pie in the sky.
Kevin Jans 3:22
It like needs, and once the threshold is the need, and the objective is the want. And then you and I’ve talked a lot about what’s happening when you’re reading a requirement, you’re trying to document the need and the want.
Paul Schauer 3:33
The downside the challenge here, the hard part is for the government to write evaluation criteria that make it clear how much that objective performance is worth to them.
Kevin Jans 3:46
How much are you willing to pay for that?
Paul Schauer 3:48
For industry if I can exceed this threshold? And since price is always a factor in competitions, how much should I exceed it by if it increases the cost? Or price? It’s a challenge.
Kevin Jans 4:04
So let’s go through a couple of examples to show just how difficult this can be but it is worth doing. And house hunting example. Right so the threshold I moved here in recently to the threshold was we want to do house within 30 minutes of our kids new school. It’s not the only reason we moved but it’s there’s a parameter I set up well, this set up a lot of options. There’s a lot of houses within a half hour of their school within your in Clearwater, Florida. The objective so it was a threshold 30 minutes 30 minute commute. The objective was within 10 minutes of the school. Wouldn’t it be cool to have a really short drive? And what this allows for is the realtor says okay, I know what the parameters are 30 minute range, which is a huge distance from the school or 10 minute rage. In the end, we found a house that’s only seven minutes away. So it’s actually worked out really well. If you miss a couple of the traffic lights, it’s still a little less than 10 Minutes. But we need to be willing to give up some features. For example, square, the house is a little bit smaller, the larger the lots aren’t as large in this particular part of town. But we were willing to trade that off, in order to have this closer location to meet our primary objective of being within 10 minutes actually, we even exceeded exceeded that. The idea is we set this objective upfront. And we said this is our main objective is to be closer. And we set that as the standard by which we search for houses. So the other elements, square footage, etc, etc. How many floors it was, those were easier to trade off, because we knew what the primary goal was based on setting the threshold and objective.
Paul Schauer 5:43
And I’m sure you had to set limits on many other parameters such as price range, it’s got to be within your budget, it can’t be more than this, or it’s got to be here, but it can’t, the backyard can’t back up to the landfill. Right? There’s there’s some things that were non tradables, probably in the equation, that you had to actually have that conversation with a realtor to define all of the variables that you are willing to trade, which ones were non starters, and which ones were tradable, and square footage a lot size are tradable,
Kevin Jans 6:17
Helps to to have other factors. But at the end of the day, their primary objective was we want to not have you know, an hour round trip for our kids to go to school.
Paul Schauer 6:27
Now while the government might may be looking to buy houses or lease houses where you want to be within a few minutes of something, that’s a more personal example. Let’s shift to another example that really can illustrate how challenging this threshold versus objectives thing can be for the government. Let’s talk I’m buying a vehicle and I care about miles per gallon I care about fuel efficiency. So I set a threshold of this vehicle must get at least 30 miles per gallon. But the objective what it really like, is 100 miles per gallon from the industry side. What if I can propose a hybrid vehicle that that runs on on gasoline and has batteries. And it easily passes the threshold and meets the objective at 100 miles per gallon, but it cost twice as much as a pure gasoline car. Is that twice as much worth it to the government? I don’t know we haven’t talked about that yet. I don’t know whether to bid something that just meets 30 miles per gallon, or whether they’re gladly pay twice as much for 100 miles per gallon. Or to complicate it even more. What if I can propose an all electric car that gets infinite miles per gallon? It doesn’t use gallons. It doesn’t use fuel?
Kevin Jans 7:56
Doesn’t doesn’t use gasoline as its fuel.
Paul Schauer 7:58
Right there you go there. Yeah. That sounds like a wonderful thing if the government is interested in not using as much gasoline. But how do we charge it? How do we fuel this this car, if it’s not gasoline, if the government’s infrastructure is all set up for gasoline fueled vehicles, this all electric car solution blows away their objective, it’s great, it’s affordable now, but it still doesn’t meet their requirements. And if they haven’t specified all up and down the line, all the many things that are important to them, you might get if you release this, this RFP, you might get all proposals for all electric cars and not be able to use any of them.
Kevin Jans 8:44
Right? What are the key factors that makes this work is you have to limit the number of things you’re having an objective and a threshold on frankly, I would recommend only one thing. The main thing is the main thing, only one variable can move. Because to your point, Paul, if you don’t tell them, I want a gas car that gets 30 to 60. If you say I want a vehicle, well now you’re making the offer, do all these mental gymnastics to figure out that they want electric car. But if you say baseline standard, I needed to be a gas powered car that I can feel at my fuel depot, they can call this many people, you need to give some set characteristics and then give them one variable to tinker with. But to your point, if you have like six or seven variables that all have objectives and thresholds, it’s going to be a zoo.
Paul Schauer 9:29
So providing offers a range between the threshold and and the real goal. The objective gives government and industry the opportunity to to to deliver or receive something above that good enough level you might if you just give a threshold just give us straight requirements, you might end up with something that is far below what the commercial industry is now using because it’s easier and cheaper just to give you that so you really liked Got flexibility, but it also allows a contracting officer to say it can’t be below this level.
Kevin Jans 10:05
And to your point using the car, right? If we don’t give a range, one of the vehicles gets 31 miles to the gallon, and one of them gets 40 miles to the gallon. But if we didn’t say that we value that range, it’s really hard to say we want to award to this company because their car gets better gas mileage, right? You didn’t say that we’re sensing you cared about. And of course, what everybody would have given you 40 miles to the gallon, if you told them all that range. So that’s why this works. But to your point, you have to kind of know where the where the customer sits in that range? What does what’s the priority for them?
Paul Schauer 10:35
How much credit does the offer get for exceeding that floor for exceeding the threshold? It’s it’s hard to define that and it’s hard to communicate it.
Kevin Jans 10:44
We had one the distance that the the the vehicle could go was it wasn’t fuel economy, this was just straight my how far it could go range. And we said we set a specific range, and no one met it. One one company came with within like 10% of it. But no one actually met it right. And so looking back, if we had set it as a threshold to say, Okay, we want it to go 300 miles, but we can live with 250. If I had if I had written the RFP that way, we wouldn’t have the problem that we had, which was no one met the 300. And I literally couldn’t afford a contract. And we had to start over, which is looking backwards, like oops, but because I wrote the requirement so that you had it was a binary you have to meet this, it led to the problem of I have an unbelievable contract, because anybody who, who if I awarded the contract to somebody who hadn’t met it, everybody else could have protested. It said, Well, I didn’t meet it either. Why did you give it to me said no, we’re down this black hole.
Paul Schauer 11:43
And that’s, that’s a problem. Even if you just have a straight requirement, right? You could have set the requirement higher than anybody could meet
Kevin Jans 11:50
That out of ignorance, like we researched it, we just didn’t have enough context. There you go on how hard it was to meet that range.
Paul Schauer 11:58
Right. And any of them could have said, well, we could make your 300 miles but we have to redesign all these things. And it’s going to take like six months longer and cost twice as much. And then you’re like, No, I don’t want that. I just want these things delivered. And actually, you know, 290 Miles is fine. That’s close enough to 300. Right? So government folks, it’s, you have to be sure that that your evaluation criteria specify what is most important to you? Is it that that that range that 300 Miles is the technical performance, is it the price is the schedule, because meeting the objectives versus your thresholds may require more time or money, an industry needs to understand how to win what what is most important in order for you to get proposals that meet your mission needs. Alright, as you might guess, what we’re talking about today occurs during the acquisition time zones, this is mostly early in the acquisition time zones. In the first zone, the requirements zone, this is where the customer sets the initial range. Then you move to the market research zone, where the contracting officer and the customer are getting feedback from industry about the range, they’ve said that is the threshold and objective range doable? Is it appropriate? Is it possible, what what are all the trade offs that happen when I move from delivering something at the threshold to delivering the objective, what happens to price, what happens at time, what happens to all the other things that are impacted by it. And like he said, Kevin, Lord help you if you have multiple thresholds and objectives, that all you have to trade off all of all the moving parts between those, when you get to the third acquisition times on the RFP zone, this is where the government has released the final RFP. And at this point, industry has limited ability to impact it. Because it’s a very formal process to modify the Request for Proposal after it’s been released, the government can still change it, but they don’t want to. And it takes a lot more time and energy.
Kevin Jans 14:06
During the selection zone in the final zone, where the selection of the winner is made. Now that range is etched in stone, the government can’t go back and change the threshold and say, Well, I know we asked for 300 miles. This is the red just told, we asked for 300 miles. But yeah, I think we can just award for 285. I can’t do that during the selection. So all the other zones, I could do that. But I can’t do that here. It’s too late to change the threshold objective, which again, is why it’s so important to have all this context ahead of time because you get trapped into a solution you can’t actually execute.
Paul Schauer 14:38
Yeah, once you’re reading the proposals, you can’t say Oh, well, I really liked this. I’m going to award to this even though I didn’t tell anybody else that this is something that I would like to late you got to have that conversation well before that.
Kevin Jans 14:50
And that’s a fundamental difference in how how we as humans often behave and how business the business sales, our business, the government sales. This is one of those moments people realize Why can’t just change my mind at the end? No, you can’t.
Paul Schauer 15:02
Yeah. Business to business. If you find something you like better, you can just say, oh, yeah, I like that a lot better, I’m gonna buy that.
Kevin Jans 15:09
There you go. 25 Miles is fine, off you go, you can’t do that
Paul Schauer 15:14
Real time I can change my mind. Not like that in the government world.
Kevin Jans 15:20
If you’re not familiar with the acquisition and execution time zones, we cover those in episode number three and episode 372. respectability.
Paul Schauer 15:28
Alright, let’s get a little FAR let’s get a little Federal Acquisition Regulations into our podcast, it wouldn’t be a contracting officer podcast without it far 7.105 Is the contents of a written acquisition plan subparagraph, a four of 7105 is capability performance, and it says specify the required capabilities or performance characteristics of the supplies or the performance standards of the services being acquired, and state how they’re related to the need.
Kevin Jans 16:00
This this language is broad, I think that’s on purpose, because it leaves plenty of room to define that required capability, including using thresholds and objectives.
Paul Schauer 16:13
Right, it doesn’t specifically say give a range doesn’t specifically say thresholds objectives. But it doesn’t say you get can’t have it, which as we all know what the far if it doesn’t say you can’t do it, then you can do it.
Kevin Jans 16:25
And this language requires help people understand what you’re what the characteristics of what you’re buying, you can have a range around that you can have a range of we we need it to weigh no more than 10 pounds, but we want it to weigh eight pounds. That’s a great explanation of what you want.
Paul Schauer 16:45
And the important part is the last part of the of the sentence state how how they’re related to the need, if you explain why you want it to weigh 10 instead of six or four, or whatever you just said, offers understand what is more important, right? Why is this part more important than that? Oh, now I know how to bid. Now I know what you care about more, because I understand what you’re getting at. It’s not just a number. Now I understand how you’re going to use it. Yeah, we’re also government folks, stating a threshold an objective range allows you to set a floor and potentially get upside that you didn’t realize was available.
Kevin Jans 17:28
Now the challenge is that you need to make sure that the government customer is willing to pay for that additional additional capability. Like like I said a few minutes ago, how much are they willing to pay for what they want, versus what they what they need? You know, this means the customer needs to be the customer, the user, the government customer, we talk about those in the three deciders, this is the person who wants the thing or the service, right? They need to be able to articulate why this higher performance level is worth the price. And like you’ve said a couple of times, that’s not always easy to say what based on our need, I’m willing to pay 10% More for something that goes 20% faster or weighs half as mch..
Paul Schauer 18:09
How do you translate that to the evaluation criteria? Because you’re, you’re not likely to be able to say I’m willing to pay 10% More for this much more, right? It really gets squishy to describe those trade offs.
Kevin Jans 18:24
Without a clear understanding of how or why to pay more for the capability. Do you end up potentially you could end up regressing back to well let’s just buy the cheapest one. So this is a slippery slope, because you can say, well, I can’t tell you why we should pay 50% More for this product. So therefore we’re not going to well, from the Offerors perspective, they’re like, Well, if you weren’t going to be able to define how you’re going to pay me more, I’m wouldn’t have bid a better solution. See how this gets really squirrely.
Paul Schauer 18:54
Using a threshold an objective strategy comes with a cost. There’s trade off in time and complexity. For your selection process. It’s much easier to specify a requirement and evaluate whether proposals meet that requirement than it is to do all the balancing of trade offs that come with using objectives on on top of thresholds.
Kevin Jans 19:18
So what do we do? How do we fix this? How does the government team deal with this? Well, you and I have said many times talk to industry communicate with folks get some understanding of what the range should be. If I had talked to a couple of the manufacturers, I probably would have realized that that particular 300 mile range, I should have made it a gap of 250 to 350 or something right? We were kind of guessing we did some market research, but we were kind of guessing. And so the lesson here is don’t guess talk to industry. I also ran into this once on a service contract where were we just copied the same thing that we did on the last source selection and said this is the standard and we ended up getting way too many offers, because the range was too big because we didn’t ask industry Hey, as is the capability of this particular profession changed? Turned out it had. Oops, right. So we got so many proposals as a result. And if we had talked to industry more, we wouldn’t have gotten as many and it wouldn’t take nearly as long ago where that contract.
Paul Schauer 20:17
Yeah, help industry understand what is most important to you, and get their feedback on those trade offs to help narrow the range or shift the range mate, maybe industry is capable much more than you expect. And you can set your threshold higher, or maybe they aren’t capable, quite what you hope. And you need to set the threshold in the objective lower, get that feedback up front.
Kevin Jans 20:39
Another way that government can minimize the risk of industry, not understanding the nuances of the trade offs between the objective and threshold is to do a downselect. The we talked about that in episode 322. The idea is the first stage includes how would you if we solve this, get an understanding of where they sit in that range. And then the second stage, you can talk about price and do the trade off them. But you break them into two pieces that makes this easier to digest. Skyway helps our customers with all this kind of stuff.
Paul Schauer 21:08
Last thought for the government folks that are transition as right to what industry cares about the purpose of the request for proposal, the purpose of the solicitation is so that offers know how to win. They know what you want, they know what will satisfy your requirements. So they can write a proposal that wins without a clear understanding of the trade offs you’re willing to accept of what you will pay more for or what you will wait longer to have delivered. Industry doesn’t know how to win. The last thing industry wants is the government have sticker shock? When they say, oh, yeah, here’s our threshold. Here’s our objective. That signaling, we want more than thresholds, right? And then you bid more than threshold and they say, Wow, we can’t afford that. We’re just gonna award to the company that met the threshold. And you could have met the threshold, but you got confused by the objective side. That’s why I generally hate thresholds and objectives. Not not because there’s anything wrong with thresholds that are objectives. But because the government usually has a difficult time explaining how to when this gets much easier if there’s actual dialogue back and forth, about all of the trade offs.
Kevin Jans 22:24
So what do we do about it? Again, yet another reason to start early, understanding what the government’s requirement would be during the requirements zone and market research zone, and engage with the government. And this goes both ways, government folks, this is why it’s important for the communication to happen. Because the what and why of what those thresholds and objectives could be, is critically important. Having a clear understanding of like you said, the trade off that government is willing to make on that price, or anything else for that matter, is critically important. Because by the time the range is posted on the solicitation, in the RFP zone, to entire zones have happened. And the impact of that, of the of an offer, like you said, guessing that you want the objective, when in reality, what you wanted was the threshold, it’s it’s catastrophic to the offer, because they’re not going to win. So the second thing industry can do is educate that government customer. Assuming you’re you’re shaping an opportunity to you know, your government customer, you’re talking to that customer, the program managers already. Let them know what the range could or should be. And we talked about this in Episode 305. And the educated customer episode, because government probably doesn’t know I didn’t know Don’t Don’t tell me is the contracting officer to know what the range should be. If this is what you do for a living, you’re the best person to inform government and government needs to listen, because otherwise you’d do it. I didn’t have to do a solicitation twice.
Paul Schauer 23:51
This is shaping right? If you do this, right, you can shape the reigns in the solicitation, so that only you can meet meet their age or only you can meet it at the at the right price, right? If you know, your competitor can only go 290 miles if their range of their vehicles only 290 miles, you want to shape that range to, you know, the floor 291 miles range with an objective of 300. And then you win, right? This is shaping and government folks are sometimes afraid of that. Right? Are they being are they being played by industry? Are they are they getting getting kind of fooled into creating a range that drives it to just one and they won’t have real competition?
Kevin Jans 24:35
It wouldn’t be awesome. From as a contracting officer. I wish I’d have seen it this way. If we could drive the range to really only three companies can bid on it. Yeah. That mean that’s the ideal situation is you’ve got it to where there’s there are a few companies that can hit between 290 and 310. And those are the three that bid. And now you’re really comparing apples to apples. You’re getting to three best companies where if you made the range from 200 to 500, who knows is going bit, right. But you don’t get that context without talking to people. So yeah, they’re each trying to make it so the only they can win it. And you’re playing this little sort of game of chess to to get it. So there’s more than one but not 50, you’re looking for the best players for the solution. And that’s why this, again, it’s a thinking job.
Paul Schauer 25:19
You really have to know the industry, you really have to know the players in order to get to be that focused with it. To wrap this up for industry, folks, be sure that you understand the government priorities and how you can win, it just may not be worth bidding, if they’re super price sensitive. And in order to exceed the threshold, it comes at a higher price it you know, you can exceed it by far, but it’s going to be a lot more expensive, and then you’re going to lose or vice versa. If you’re not talking to the customer, if the customer is not communicating to you, and you don’t know how to win, you may not want to even bid.
Kevin Jans 25:58
And this guy with a team helps with all this side of it, too.
Paul Schauer 26:01
All right, Kevin, we’ve met our threshold of time for a podcast, let’s Our objective is to be our objective was even faster than this shorter than this more focused than this, but we’ve definitely hit the threshold.
Kevin Jans 26:14
Our threshold is 30 minutes or objective is 20. So I think we’re between the two right now. Basically, the threshold is good enough, like our listeners that we talked to a lot that anything over 30 minutes, or it’s too much, it’s longer than their commute, right? Whereas 20 minutes is perfect. Is it the big takeaway from this is that this may not be the most expeditious acquisition strategy. It’s not going to feel fast. So roll through this, this is not as easy as saying just meet all these basic requirements, check boxes and go right this is not going to be that Expeditious, however, it’s going to be a lot more effective. And you’re, you’re you’re more likely as the government to get better outcomes when you have a range for offers to bid into.
Paul Schauer 26:57
I’ll say it’s very possible. Maybe I’m not gonna go as far as more likely, but it’s very possible in the right situations. This is incredibly helpful, right?
Kevin Jans 27:07
Yes, that well said when used properly. You know, what, what’s that thing of, with great power comes great responsibility. This is one of those, if you do it wrong, it’s really going to blow up in your face.
Paul Schauer 27:16
If you’re going to do anything other than set a threshold, if you’re going to do anything other than state a requirement that offers have to meet. Make sure that everyone on both sides understands the intent of the objectives and what you’re willing to trade to get to the objectives. If you do all that you’ll have better outcomes. All right. With that, I’ll talk to you later, Kevin.
Kevin Jans 27:40
I’ll see you Paul.
Paul Schauer 27:42
Okay, that’s it for another episode of the contracting officer podcast. You can find podcasts in playlists sorted by topics on our site at Skywayacq.com/cop for contracting officer podcast. As always, thanks for listening, and we’ll see you next time.