Add custom apex:inputFile icon/image

This question is more related to CSS. You can achieve this by CSS.

Just provide you inputFile tag a class like:

<apex:inputFile value="{!attachment.body}" filename="{!attachment.name}" id="file" styleclass="fileType"/>

And apply the CSS from this link:

html {
  margin: 30px;
}
.fileType {
    display: block;
    position: relative;
    width: 200px;
    margin: auto;
    cursor: pointer;
    border: 0;
    height: 60px;
    border-radius: 5px;
    outline: 0;
}
.fileType:hover:after {
    background: #FF1111;
}
.fileType:after {
    transition: 200ms all ease;
    border-bottom: 3px solid rgba(0,0,0,.2);
    background: #000000;
    background-image:url('http://i.stack.imgur.com/CVpp3.jpg');
    text-shadow: 0 2px 0 rgba(0,0,0,.2);
    color: #fff;
    font-size: 20px;
    text-align: center;
    position: absolute;
    top: 0;
    left: 0;
    width: 100%;
    height: 100%;
    display: block;
    content: 'Upload Something';
    line-height: 60px;
    border-radius: 5px;
}

Or you can play with your own CSS too and you can provide your own image url for background property of CSS.