Clearly there were Marshalls in the Ickleford area and the 1881 census shows a Willoughby Marshall (56, born Ickleford) and his brother James Marshall (52, agricultural labourer, born Hitchin) living in Ickleford - and they might well be the brothers to the Charles Marshall that you found on the IGI - and you may find their baptisms (and maybe other siblings) by looking on familysearch - a quick glance showing that the uncommon given name Willoughby seems to occur frequently in the Marshall family in the area.

However Marshall is a common surname and at the 1881 census there were over 50,000 people with the surname, including nearly 1000 people recorded called Charles Marshall, 74 of whom were born within 5 years of 1826. Allowing for infant deaths, emigration, etc., there were probably about 10 children called Charles Marshall being born each year around 1826. James is a commoner given name than Charles - with about 40 children a year being born about 1852. Joseph is about as common as Charles. If the information is correct your Charles H. Marshall, son of Joseph and Jane is clearly not the Charles Marshall, son of James and Ann of Ickleford. The surname Uttley is virtually unknown in the county.

Unless there are definite evidence - such as  that when living in the US Charles still owned property in Hertfordshire - or strong circumstantial evidence such as that the James B(?) Marshall born circa 1851 was another James Baker Marshall and/or Charles had another son which he called Willoughby - you would be well advised to look elsewhere,

I suspect, from your mentions of the IGI, that you have been putting far too much emphasis on the results of negative searches in this incomplete index - see The Limits of Familysearch. If Charles married in England and/or James B Marshall was born in England the obvious place to look is the GRO indexes of births, marriages and deaths, which is almost certainly available on microfiche or microfilm at a genealogical library near you - possible an LDS Family History Centre. If positive this would allow you to purchase the relevant birth or marriage certificates. If you were not aware of this resource you should have a good look at the How to ... pages on this site, the beginners guide on GENUKI, or read a good book on researching your English Origins.

You may also find  My Ancestors Emigrated from Hertfordshire useful, and should be aware of the kind of problem highlighted in What did you Ancestor call himself.