Network-based Intrusion Detection Systems (IDSs) such as Snort or Bro that have to analyze the packet payload for all the received data show severe performance problems if used in high-speed networks. Recent research results improve pattern matchers based on efficient algorithms or using specialized hardware. We approach the problem in a completely different way by considerably reducing the amount of data to be analyzed with only marginal impact on the detection quality. Dialog-based Payload Aggregation (DPA) uses TCP sequence numbers to decide which parts of the payload need to be analyzed by the IDS. Whenever a connection starts, or if the direction of the data transmission between peers changes, we forward the next N bytes of traffic to an attached IDS. All data transferred after the window is discarded. Our analysis using live network traffic and multiple Snort rulesets shows that most of the pattern matches occur at the beginning of connections or directly after direction changes...