Do you have any brands in mind? In the bagged tea with cellulose tea bags that I've bought, they are typically held together by just a staple. I've found this to be the case since I occasionally empty the tea from the teabags into my percolator.
Pillow style tea bags are fully enclosed paper tea bags with no string or tags, and have a crimp around the rim. I never thought about them using glue, but it makes sense. Some brands I know of that use them are Taylors of Harrogate, Celestial Seasonings, Republic of Tea.
Edit: Also some tea bags for loose leaf tea like t-sac or finum brands have that crimp on the edge. However, t-sac confirms they use glue, but finum specifically claims to not use glues, so maybe it is jumping to conclusions that all bags of these sort do.
