Solutions and Problems

My Recent Posts

 

There is a phenomena in human society that IMO has extremely important implications. It has been stated perfectly by  G. K. Chesterton   in “The Point of a Pin,” The Scandal of Father Brown (1935):

 

“It isn’t that they can’t see the solution. It is that they can’t see the problem.”

 

It is my experience that this confusion in society is so ubiquitous and so little recognized that it is a major obstacle to both personal and societal progress.

 

Indeed John Dewey,  Logic-Theory of Inquiry, pg. 108,Henry Holt and Company (1938), stated the following

 

1) “It is a familiar and significant saying that a problem well put is half solved” 

2) “Without a problem, there is blind groping in the dark.”

 

Dewey used the terms “problematic situation” and “well put problem” in the development of his analysis. Let me give a specific example from my personal experiences. 

 

About 15 years ago there was this big push  in cybersecurity to develop Intrusion Detection systems, namely computer code that could look at institutional computer network traffic and assess which packets were intrusions by external threats.  An i  agent  (with access to funds) came to us for help and after an hour  tell us of the difficulties they were encountering, said something like 

 

“we need to develop some computer methods to detect intrusions and we are seeking help”

 

There then ensued an hour of solution methodologies proposed and discussed. 

Near the end, I raised my hand to speak and said:

 

 I can do that, it is very simple and requires no expense, and is lightning fast and efficient, He said how? I said: simply call all packets intrusions. The room laughed as did the guest. He said, you cant be serious, the false alarm rate would be huge and our agents would be overwhelmed with investigations. I said, oh, I can solve that problem perfectly also. Simply call all packets “non-intrusions” and again the room laughed but this time the laughter was a little nervous. Now the agent become serious and said  then we would  have too many undetected intrusions.

I responded , now you speak of “false alarms” and  “undetected intrusions”. Do you  also have computational concerns like how fast this system operates? What kind of assumptions are you willing to make about the nature of historical traffic with the traffic for the deployed system? What type of “training data” might  one have?( Training data is data where we know if a packet is intrusion or not). 

 Will the development have  have access to domain experts?

 

To not get bogged down with the specifics of this specifc problematic situation, let me tell my readers what is the hallmark of a well put problem, according to Dewey;

 

 If Tom  produces a  solution and Mary produces a solution, then there is a pre agreed on operationally relevant method for evaluating which solution is better. 

 

One of the hallmarks  of a non well put problem is that different solutions are compared with the “view graph metric” namely by the coolness of the visual display. Another common method is to rank solutions by the prestige of the problem solver.

 

In the following weeks and months a working group developed to address this problemlematic  situation. However, it became very clear that  a formulation of a well put problem  or even a  “performance metric”  was anathematic to all but very few. The remainder had great ideas for solutions. This  struggle the minority lost in quick order. But our institution was successful in getting the agent to fund a multimillion dollar research effort on intrusion detection. After 10 years,  they have many solutions and no clue on how well any of them work or will work when deployed in a new environment. There is IMO a blind groping in the dark.

 

 

This is but a small example of my experience in science -there are many nonscientific examples. As a warning,  once you become aware of this phenomena, you are never the same. You develop a sixth sense:

 

I see “solutions” everywhere

 

 

 

Comments

The Burghal Hidage Added Mar 11, 2018 - 11:51am
So there you were,left like Graham Chapman's title character in the Life of Brian :  " Look, I haven't got all the answers! You'll just have to figure it out for yourselves!"
 
 
Dave Volek Added Mar 11, 2018 - 12:02pm
I have had similar life experiences. I throw up some interesting possible solutions and eyes in the room start rolling. In their minds, I can hear: "There goes Dave getting all analytical again."
 
One part of this essay I like is that whatever solution we decide on, it should have a measurable result of some kind.
Pardero Added Mar 11, 2018 - 12:04pm
Mustafa,
Correct me if my simple blue collar translation is wrong:
Identify the root of a problem, not the symptoms, and tailor a solution that directly addresses the root. 
Dino Manalis Added Mar 11, 2018 - 12:15pm
 Problems and solutions have to be spelled out for everyone to understand!
Neil Lock Added Mar 11, 2018 - 12:16pm
Mustafa: Great article (when was the last time anyone on WB got 4 likes in the first 7 views?)
 
As I suggested in my recent article on the precautionary principle, the underlying "problem" may well be the inversion of the burden of proof. It should be the responsibility of those promoting the existence of a problem to define the problem fully , and to prove that it really is a problem. The next stage, as you say, is to provide an objective metric for how good a proposed "solution" is. Without this, all "solutions" put forward will probably be destructive.
The Burghal Hidage Added Mar 11, 2018 - 12:28pm
The electric power industry is plagued with this, particularly in the areas of cyber-security. Most of their organizations are structured in a manner that when they discover that they "don't know what they don't know", the typical reaction is paralysis
Mustafa Kemal Added Mar 11, 2018 - 12:29pm
Dave, in my line, we dont find solutions until we have a good grip on the problem.  Then, it is our experience that Deweys other statement is quite true:
“It is a familiar and significant saying that a problem well put is half solved” 
Mustafa Kemal Added Mar 11, 2018 - 12:51pm
Pardero, close, but not just the root, but the fundamental things of concern  and the fundamental assumptions that one is willing to make. 
 
BTW, in my experience blue collar minds understand this better than others. If I ask you if you can weld me a truck rack and you  have welded twice in your life, you could tell me "sure I can do that"
and you would be correct, and I may get just what I deserved. But if I say I want truck rack with "this type" quality welds, of "this" structure, and within "this" costs, then you make think twice about your solution 
and recommend me somone who may able to provide a quality solution.
 
 
Here is a good example: illegal immigration is a problematic situation.
But what is the problem? Is it the amount of crime? Drain on resources? Social division? Contribiution to cheap and physical labor? Displacement of US citizens jobs?  Costs of implementation. 
 
If this is a "problem" that one would like to truly solve, then we should formulate this problem and agree on quantitive aspects of  issues like the abovementioned concerns. When ( if!) this is  accomplished, then rational debate could ensue, not according to everyones favorite underlying narrative, but only with respect to the agreed on objectives.
 
 Then, when solutions  are implemented, say e.g. The Wall is built, then one would be in a position to assess the quality of an implemented solution on a rational basis. 
 
Mustafa
 
Mustafa Kemal Added Mar 11, 2018 - 12:54pm
Neil, we are in sync except for one point.  
re; "and to prove that it really is a problem. "
depends on the realm. If a customer comes to me  with funds, I dont really care. Or course, if the problem is so trvial that I cant in good  conscious accept payment then I will reject the work to maintain reputation.
Mustafa Kemal Added Mar 11, 2018 - 1:03pm
Burghal, re;
“ Look, I haven't got all the answers! You'll just have to figure it out for yourselves!”
 
Not really, I may of the answers, but i need to know what the problem is first. But, there is one problematic situaion that I do know the solution to:
 
“Always Look On The Bright Side of Life”
 
https://www.youtube.com/watch?v=WoaktW-Lu38
Neil Lock Added Mar 11, 2018 - 1:33pm
Mustafa: If a customer brought me a problem that I had good reason to believe isn't real, I'd probably say something like: "Would you like me to analyze the problem first, and report to you on that?"
Mustafa Kemal Added Mar 11, 2018 - 3:11pm
Neil, 
re:
"If a customer brought me a problem that I had good reason to believe isn't real, I'd probably say something like: "Would you like me to analyze the problem first, and report to you on that?""
 
If that is what you meant, we are in full agreement in substance but not style.
 
It is my experience that almost all customers have little clue about what a well formulated problem is, so when they come to us, the first thing we require is that we get to work on that. These things must be done delicately (a la the wicked witch of the west) since you can easily bruise the customers ego with brash versions of the quest. So, you begin slowly and try to get them to discover what it is that they actually want.   With a good customer this can be difficult but with  bad one, nearly impossible.  In the good cases this will dovetail with what we know that we can actually do, or think that we could reasonably develop  in the timescale of the effort. The bad ones are the ones who are not truly invested solving the problem ( think government employee here). or just not suited to thinking about such problems.
 
But doesnt just apply to technical things. It can be applied to family problems both personal and financial.  
 
 
Mustafa Kemal Added Mar 11, 2018 - 3:14pm
Burghal, re:
"The electric power industry is plagued with this"
Indeed, as are most industries where there are vested interests in maintaining a status quo, or the new development culture has an intereste in maintaining the "view graph" performance analysis, which often can be used to impress despite substandard solutions. 
Benjamin Goldstein Added Mar 11, 2018 - 3:19pm
What was the result of the research effort? Was a solution developed from the group who laserfocused on the problem definition or was a solution taken that proved to solve multiple problems at once?
 
I mean, I'm not sure if your brain storming colleagues were neccessarily on the wrong track. Particularly the field of computer security is an area where you cannot easily mark the problem, developing a measure as to how well the solutions work is even more difficult (do you have such a measure with regards to IDSs?).
 
Maybe, you mean that the offsets were not properly discussed. Different systems need different levels of protection. What is essential for one user (group of users) might be less vital for the others and vice versa. Yet, all solutions for protecting various aspects come with a cost in terms of computational capacity or interrupting the user.
mark henry smith Added Mar 11, 2018 - 3:25pm
Well that sucked, I was almost done my comment and someone logged me off when I had plenty of time left.
 
Here's what I said, I think.
 
In your above example the problem is that an incoming package cannot be known to be an intrusion until it does something intrusive. Many problems in this area can sit for long periods of time waiting to do something nefarious when triggered. I call this the moving van problem. When putting things on a moving van, the van is weighed once loaded. To maintain scrutiny of all packages for as long as they were on the van would be costly.
 
So incoming packages could be labeled by the sender and packages of similar identity could be compared to see if they are of similar size since it would make sense that two packages doing the same thing would be of similar construction, and the oddly weighted one could be watched. Or abandoned by the side of the road. The entire van could also be weighed regularly to see if it was picking up weight from unknown sources.
 
In the modern world we have problems that we know are problems, but people refuse to call them problems, such as the automobile. The automobile is a huge problem for the world and is only getting worse. It's not just the road building that decimates areas, or the emissions, or what regular, heavy traffic does to environments. It's also higher salt concentrations in streams and rivers and ground water due to the applications to melt ice and snow.
 
What makes a problem invisible? Convenience. If something makes our lives too convenient, it can't be a problem, it's an asset no matter how much damage it causes. Nice post Mustafa.      
Mustafa Kemal Added Mar 11, 2018 - 3:40pm
Benjamin,:
Thank you for ratcheting this up a bit.
re:
“What was the result of the research effort? Was a solution developed from the group who laserfocused on the problem definition or was a solution taken that proved to solve multiple problems at once?”
 
Neither, never was a problem formulated, but the effort provided many solutions, that were never “proved” to  be solutions of anything.
 
re:
”I'm not sure if your brain storming colleagues were neccessarily on the wrong track. Particularly the field of computer security is an area where you cannot easily mark the problem, developing a measure as to how well the solutions work is even more difficult (do you have such a measure with regards to IDSs?).”
 
Just because it is not easy does not imply you should not work at it. It is a false dichotomy to go from “it is hard” to “we dont care” 
 
 
“Maybe, you mean that the offsets were not properly discussed. Different systems need different levels of protection. What is essential for one user (group of users) might be less vital for the others and vice versa. Yet, all solutions for protecting various aspects come with a cost in terms of computational capacity or interrupting the user.”
 
Now you are getting somewere.  You are describing the vector version of a well formlated problem, namley it is not possible always say that one method is better or worse. Not all problems have scalar objectives, so one solution may do well in one criteria and poor in others. Nevertheless,  putative solutions can be assesses quantively. 
And if well formulated solution methods sought.
 
Another example of this kind of thing is that a system trained in one environment may do well in one and not in the other. 
This has to do with the assumptions on the relationship between the training environment ( an institute possibly) and a deployed environment (Afghanistan?)
 
Benjamin Goldstein Added Mar 11, 2018 - 4:13pm
Mustafa: So some solution was taken. It might well be not worth the effort of your team and only a standardized external gimmick, but a IDS is in place.
 
I mean with 'not easy' the verge of impossible. You cannot know the attacks beforehand and the only shield against all attacks is to get rid of the system itself (as you have suggested keeping out the input altogether).
 
There is the basic problem in security to define damage. What change is a harmful change?
 
IDS usually works on 'experiences', on logs. Any change that is considered to affect a vital area prompts an approval seeking mechanism to a user with enough authority.
 
May I ask: How did you define 'damage'? Or was there a particularly damage most worthy of avoiding? If so the solutions can be measured on that. For, example, some part of the system must never be interrupted. The downtime after the security measure was taken can be logged. I assume as this is the main damage, you have a record of the downtimes before.
 
So I guess the problem definition comes on top of a damage definition, which in turn comes on top of a prioritisation of what is vital in your system.
Pardero Added Mar 11, 2018 - 5:07pm
Thanks Mustafa,
It is a bit above my pay grade but I try to glean bits of wisdom.
Mustafa Kemal Added Mar 11, 2018 - 8:04pm
mark henry smith, thanks for writing.
re:
“In your above example the problem is that an incoming package cannot be known to be an intrusion until it does something intrusive.”
 
I used words as place holders here. What an intrusion actually means to the customer and with what types of data that are available that might be good make that assessment will all have to be determined.
re”
“So incoming packages could be labeled by the sender and packages of similar identity could be compared to see if they are of similar size since it would make sense that two packages doing the same thing would be of similar construction, and the oddly weighted one could be watched. Or abandoned by the side of the road. The entire van could also be weighed regularly to see if it was picking up weight from unknown sources.”
 
Now I think you are talking solution methodologies.
re;
“The automobile is a huge problem for the world and is only getting worse.”
 
This is a very good example of a problematic situation. When i first read your statement  I thought, problem? How will I get to work? How will I go visit my mother? So, I have some benefits and you have some costs. But we still dont have a real problem such as: what changes in our society would bring the costs down by at least this much but bring the benefits down at most this much.   Something like that
 
Mustafa
Doug Plumb Added Mar 11, 2018 - 8:35pm
I would say that stating the problem in precise terms is 95% of getting to the solution. Usually problems are not well posed and we skip over improving the problem statement, or deciding if this is a symptom of the problem or the problem itself. Confusing symptoms with the problem is a huge problem in mainstream political discourse.
Mustafa Kemal Added Mar 11, 2018 - 8:40pm
Benjamin, good to see you again,
re:
“but a IDS is in place.”
Not where i worked and they hadnt found anything that they were happy with.
Some in the effort spun off and started companies selling such IDS.
 
re:
“I mean with 'not easy' the verge of impossible. You cannot know the attacks beforehand and the only shield against all attacks is to get rid of the system itself (as you have suggested keeping out the input altogether).”
 
If it is truly on the verge of impossible then you dont need those solutions anyway.
Now, in reality for complex problems one may not really try to formulate the uber problem, but there may be many subproblems with are formulatable and  the complex problematic situation can be thought of as a graph connecting well formuated subproblems.  Moreover, I have seen many situations where they are not really as impossible as one  thought. In my experienc the problem formulations are mostly never considered.
 
re””IDS usually works on 'experiences', on logs. Any change that is considered to affect a vital area prompts an approval seeking mechanism to a user with enough authority.”
this of course is a solution methodology, a rule based methodology.  How well do they work?
I propose that can only be assessed if you tell me what the well formulated problem is. Yes, there are many companies that sell solutions.
 
re:”May I ask: How did you define 'damage'? “
As I said, we were cut out early. I only know what they did through colleagues that worked  on it.
Generally they made some definitions of things that they are concerned with -damage or potential damage. You would think the customer would have a good idea about what they were concerned with.
 
re:”You cannot know the attacks beforehand and the only shield against all attacks is to get rid of the system itself”
Now you are really talking. This indeed IMO is the real heart of the matter. To begin, I dont expect to have perfect detection.  I would like to reduce the number of successful attacks as much as possible without overwhelming security experts with all our false alarms; e.g. constrainb false alarms and maximise detections.  The fact that the field of attacks is evolving only means that the system needs to also evolve, so now we are not talking  a static system but an adaptive system.  Now you have a time dependent “problem”
but you can use your predictions from the past along with some “selected samples” given to security experts to retrain and adapt.  Of course, none this will protect against somebody who discovers something a fundamental weakness.. These types of attacks are best fought by rule based systems built by good hackers. But we can only try to do the best we can do. The problem formulation will give us an idea what that actually means.
 
You are correct that this is a complex problem, maybe I should have begun with a simpler one.
But you have brought up many inportant issues.
 
Mustafa
 
 
 
Mustafa Kemal Added Mar 11, 2018 - 8:45pm
Doug,  Dewey put it at 50% and my experience is that you may be closer to the truth in the domains in which I worked. But the main point is, a solution may be a good solution to a problematic situation if you had formulated it, but you will never know.
 
And you are spot on about political discourse.
Phil Greenough Added Mar 12, 2018 - 8:28am
Illegal immigration is a perfect example of why this article makes no sense.  There will never be “agreed on objectives” for complex problems.  Because of that there will never be “solutions” to complex problems.  I believe illegal immigrants are a benefit to our society and that a wall in remote parts of the Mexican/American border will accomplish nothing.  Others believe the wall is the solution to most of our immigration woes and that rounding the ones already here is a manageable task.  Between those two viewpoints are thousands more opinions on the matter.  Not to mention that of all those viewpoints, none will be a solution. 
mark henry smith Added Mar 12, 2018 - 12:37pm
Phil, I humbly disagree.
 
All sides do agree on a solution, that there has to be some form of legal immigration that allows the good guys in and keeps the bad guys out.
 
After that it gets tricky in defining good and bad guys and the methods that should be used to enforce border security. But I think we also all agree that we have to be a nation of laws that can be applied equally to all.
 
I heard a sob story on NPR about a guy who's lived here illegally for thirty years working hard and being a good contributor, of course how they know this is mere speculation, and now he's being deported and how terrible that is for him and his family and I wanted to ask, let's say the guy had been a murderer who had gotten away with murder for thirty years, but had been equally good since, which would of course be mere speculation would you feel the same?
 
And of course the definition of genius isn't in finding better answers, it's in asking better questions, as Einstein said.
Benjamin Goldstein Added Mar 12, 2018 - 12:51pm
I think, your example is actually good enough to correct your mistake.
 
The way systems are secured is by heaping layer upon layer of solutions to vaguely defined to undefined problems. The fussy logic employed is that it should not cost too much.
 
It is how it's done and your computer in front of you would be a bleeping mess, asking you for some ransom money to unlock your hard drive, if people would have sit around to specify the exact problem.
 
You lock your door for no good reason. Maybe an intruder steals your stuff. Maybe a neighbour just feeds your cat. Maybe you would have no problem with one unimportant thing or another be removed from your home or that your fat gets too much food. In any event, you lock the door because you vaguely sense that a myriad of problems that you cannot describe right now might arise if you don't lock the door. So you just heap that layer of security on top of carrying your bank cards with you and leaving no notes with the PIN. You add another random layer of security by .... doing whatever. It is how it works.
mark henry smith Added Mar 12, 2018 - 1:15pm
And Ben, that's exactly how the political process works. New laws are heaped upon old laws without old laws being removed. And we just assume that the new laws are better because they're newer. Like cash bail. The old law still exists that cash bail is illegal for all except violent crimes.
mark henry smith Added Mar 12, 2018 - 1:17pm
We lock our doors because we assume we have things worth stealing, or because our landlord insists upon it. But we still leave our mail in an open box.
Mustafa Kemal Added Mar 12, 2018 - 3:06pm
Phil, thanks for the very relevant observation.
re:
"llegal immigration is a perfect example of why this article makes no sense.”
There will never be “agreed on objectives” for complex problems. “
 
You indeed bring up relevant points,. Here we have a problematic situation with "two customers" at least.  As for jumping, and I mean really jumping, to the conlusion that this article makes no sense,  that makes no sense.
 
A thought experiment. One side rules all. The legislature, the judiciary, and executive. They want to solve this nasty immigation problem and they dont need approval from the other side. 
So, they come up with a solution, a Wall. But what problem did or did it not solve? Certainly this should be easier than the mosaic you display.
re:
“There will never be “agreed on objectives” for complex problems. “

Maybe, maybe not. but maybe some of us get together and agree on a problem or an array of problems. Then we have the tools to discuss putative solutions peformand and evaluate implement solutions performance with respect to our proble. We may even find out that the other sides solutions are decent solutions to our problems. 
re:
“Not to mention that of all those viewpoints, none will be a solution. “
I dont think you read my post.  Here:
 
It isn’t that they can’t see the solution. It is that they can’t see the problem.”
 
There will be solutions. If the wall is built, that is a solution. If the sanctuary cities are dismantled that is a solution. If extraditing every illegal alien, including all of DACA occurs, that is a solution.The question is a solution of what? Or better, if we knew for what, how good of a solution would it be. 
 
There are many solutions. Those are really easy to come by. The second assertion agrees with your position 
“It is that they can’t see the problem.”
 
The difference to me is that you think it is impossible to find a problem and I do not. My experiene is that people just dont try. They simply jump to solutions.
 
Mustafa
 
Phil Greenough Added Mar 12, 2018 - 7:32pm
Dismantling of sanctuary cities and a wall will only make the “problem” worse.  I put problem in quotes because I don’t think illegal immigrants are a problem.  See what I mean about there not being any solutions?
Mustafa Kemal Added Mar 12, 2018 - 9:43pm
Phil,  thanks for giving me a chance to clarify. re:
"See what I mean about there not being any solutions?"
No, not only that, although I may agree with you that
"Dismantling of sanctuary cities and a wall will only make the “problem” worse.",  these are still solutions. They just may not be good solutions to any problem you find relevant. Indeed, they may not even be good solutions to any problem related to illegal immigration.  The promise of the wall may only be a good solution to getting elected.
 
In fact, that IS half the point of Chesterton's assertion. If you look around you find that  solutions are everywhere.
 
Mustafe
 
Doug Plumb Added Mar 13, 2018 - 5:16am
re "But we still leave our mail in an open box."
 
Mail is sacred, open someone mail and you may as well be pulling down their pants. Also, mail is where the wise conduct court.
Utpal Patel Added Mar 13, 2018 - 7:01am
I think you’re confusing the word “solution” with the word “suggestion.”  Solutions solve problems.  Most political and economic problems have no solutions.  For example, there will always be mass killings no matter what law is passed on gun control.  Hopefully because there are no solutions people don’t stop making suggestions.  Of course many suggestions can make problems worse or cause other problems and many of which make things better.  That is why we have a legislative branch rather than rulings by diktat of a single person. 
Even A Broken Clock Added Mar 13, 2018 - 10:23am
Mustafa, very interesting post and subsequent discussion. Let me share something from my work experience. In the chemical industry, we conduct process hazards reviews. These are systematic reviews held for chemical processes according to a pre-defined methodology where teams of engineers, chemists, and process operators go through and identify where there may be a problem in that there is too high of a probability of a bad event. A bad event may be a loss of containment, or an explosion, or some other circumstance that could result in serious injury or death.
 
So in these cases, the team is tasked with identifying the problem in the first place. We had to come up with a problem definition that was clear enough to persuade management that solutions were needed. Often there were two levels of solution - one which would be procedures based and highly dependent on operator actions, and one which would provide systems based solutions that did not depend upon operators.
 
Anyway, it was a very interesting situation, and often I as an engineer on the team was tasked with providing the solution after the review was conducted.
 
Mustafa Kemal Added Mar 13, 2018 - 11:57am
Even a Broken Clock, it is my experience that operations with well understood bottom lines, such as real experiences with 
"A bad event may be a loss of containment, or an explosion, or some other circumstance that could result in serious injury or death."
tend to understand this approach much better than others. To them, unlike many of us, they do not consider this "theory".
They often keep records and assess "So, since we implemented that solution, how many bad events did we experience?" 
 
We had an opportunity to do some work for the military and I was excited to engage for just this reason, but the corruption there is as bad as other parts of the US,  I was mistaken in my expectations.
 
Mustafa
 
 

Recent Articles by Writers Mustafa Kemal follows.