From Libertarian Wiki

Same-sex marriage, along with abortion, is one of a handful of issues that Libertarian candidates are often questioned about in conservative districts. There is really no pleasing the theocrats on this issue, unless you advocate laws defining marriage as a union between one man and one woman (which also, incidentally, would prohibit polyamory). So, you may as well take a strong libertarian stand by saying that you don't believe the state has any business defining what marriage is, or even issuing marriage licenses. George and Martha Washington did not get a marriage license, and most people did not need them until the mid-1800s. Marriage was considered a religious ceremony, outside the purview of the state.