Given that TreeValueProxy uses a conversion operator returning a const value of the template type, you simply need to tell the compiler to use that conversion operator, so that it doesn't try to invoke .first on the TreeValueProxy class itself.
You should be able to simplify the existing suggestions and avoid redundant declarations by using a temporary and taking .first of that. I assume the following are ultimately equivalent:
// cast to invoke specific conversion operator
auto str = static_cast< std::pair<Glib::ustring, Glib::ustring> >( row[columns.col] ).first;
// call copy ctor to make it deduce the right conversion
auto str = std::pair<Glib::ustring, Glib::ustring>{ row[columns.col] }.first;