In the two dimensional visual arts the surrounding of an image is called frame and ground. It is of integral importance to the quality of the image. It may be the most important characteristic that turns a snapshot into a photograph.
What we are trying to achieve in a photograph, or a painting for that matter, is to open a view onto the world that shows a picture of a scene at a chosen moment in time. We want to do so in such a way that the edges of the image go unnoticed. When we stand and look at what's in front of us we don't notice the edges of our vision, our eye lashes, a bit of clothing that may be obscuring the scene. Our brain frames the scene so that we make a complete image in our mind that we can consume.
The two aspects of framing the scene artificially in a camera are the straightness of the cropping and what is included, or cut off, in the image.
I feel when I look at a photograph that the straightness is the more important of these two. A photograph should be cropped to the horizon most of the time. We live in a world with horizon and most of the time are aware, subconsciously, of its position. When a photograph is cropped crookedly it is very noticeable that something is wrong. What exactly is wrong may not be apparent, but there is a sense that the image is not well made.
Cropping straight can be a challenge. The horizon may not be be visible in the image. We may take an image of a sloped scene or from a sloped scene. The distortion of the lens causes vertical lines to converge which can distract from the location of the horizon (this can be adjusted somewhat in modern digital editing software). My experience is that when I get the cropping just right, there is a bit of a mental click that the image has reached it's proper location.
I feel that the details that are cropped into or out of an image are less important, although when very carelessly done can be just as important as the straightness. In this case as well I believe that whether to leave something in or take it out feels somewhat obvious while looking at the image. The edge of an object that is of a strongly contrasting color or shape that just protrudes into the frame should be removed. Half of a car in a street scene where some bit of architecture behind or next to it is very dominant should be left in at the risk of removing some important aspect of the image.
There are clearly some types of images where the above doesn't hold true: abstracts, some still lifes, possibly fashion and nudes that are cropped dramatically for effect. I believe it holds true for the type of art photography in which I am trying to show you a window into the world I see.