Jump to content

Any IPS 4.1 high-load community examples?

Ibragim Pupkevich

By Ibragim Pupkevich
in Feedback

Recommended Posts

Ibragim Pupkevich Enthusiast

Ibragim Pupkevich

Posted
Posted

Dear IPS team,

I'm experiencing quite nasty troubles with my IPS 4.1 self-hosted community, which is quite heavy in terms of DB size (>4GB already) and daily page views (>70k).

Even for 8-core Intel Core i7 CPU + 32GB RAM + SSD it appears to be quite hard job to run IPS engine scripts, server goes down quite often running php-fpm environment.

Do you know any examples of other high-loaded communities running IPS 4.1 suite? Maybe their administrators can share some experience?

OPcache helps a lot, but still... looks like IPS suite engine is very heavy. Different from older 3.x versions :( We have faced even higher page views rate 2 years ago, when we used IPB 3.4. But we've never faced any performance troubles like we face now. And it's not DDoS or any attack-related behaviour, already checked for that.

Thx.

Link to comment
Share on other sites
tolik777 Enthusiast

tolik777

Posted
Posted

Hi!

I have DB Size over 8Gb, daily page views over 100k. Online users 1000-1500 on peaks. And 2 000 000 posts, 100k registered users. My forum is: http://forum.cxem.net/

4.1.17 very hard VS 3.4... Now, my server is: i7-6700,  64 GB DDR4,  2 x 250GB SSD...

The third version worked fine for less configuration. This version 4.1.x is slower than running on more powerful hardware.... If I off cache (I use Redis) - SQL immediately falls...

I use NGINX + PHP 7.0 FPM. I do not recommend Apache on high load. And What do you use apache or nginx? php 5 or php 7?

Link to comment
Share on other sites
mark007 Enthusiast

mark007

Posted
Posted

I had massive performance problems with IPB 4.1.17.1.

Then I disabled memcached, installed ZendOPCache and PHP 7.

Data Storage method: MySQL

Caching method: no

Cache page output for guests: no

--------

MySQL Query Cache 96MB

max_heap_table_size=36M
table_open_cache=2048
query_cache_limit = 2 M

-> now IPB 4 is running faster then 3.4.8 !!

Link to comment
Share on other sites
  • 2 weeks later...
Colonel_mortis Proficient

Colonel_mortis

Posted
Posted

I use redis along with some custom plugins to move more things into redis (which reduced response times by >30% from what I remember) on a high traffic (10k peak online users, normally closer to 1-2k online at a time), and the mean server response time is ~100ms (median is <50ms). One thing that made a massive difference was removing the Who's Online block. Sure, it makes the site look good because lots of people are online, but it had a very significant impact on performance.

Link to comment
Share on other sites
SecondSight Proficient

SecondSight

Posted
Posted

Hello ! :)

I have about 800 users online and I don't use any cache software (like xCache). My server has got 2 SSD disks, 64 Gb RAM and the processor is Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz.

My database is now 11 Gb big... With IPS Community 3.4, everything was working fine because I had a table in Memory and two other tables were InnoDB (these were the tables causing problems - If I remember well, it was sessions and core_item_markers tables, not sure).

Regarding IPS Community4, everything worked fine after I changed all tables from MyISAM to InnoDB.

I also noticed each time I upgraded to a new IPS Community version (for instance, from IPS Community 3.4 to version 4), I had Apache performance problems for some time. I suspect search engine bots to be responsible for this (anyway in the logs I found there were too many connections for bots).

Link to comment
Share on other sites
  • 2 weeks later...
  • 3 weeks later...
ExiledVip3r Explorer

ExiledVip3r

Posted
Posted

I have a community of currently ~1,500,000 members, ~1,000,000 daily pageviews, and a 9gb database for the forums.

I really want to upgrade, and I've tried twice now to upgrade my community; first to a midlife 4.0, and again to a midlife 4.1. They both ran like complete and utter crap. When not crashing the servers, they were ideally pushing 25% of the load times I got using 3.4.

Even after optimizing them to the best of my knowledge. Using the same server environment that runs perfectly smooth with 3.4 with power to spare. It has been a complete and utter farce with 4.0+ to the point that I've been forced to apologize and revert the community back to the pre-"upgrade" backups both times after trying 4.0 and 4.1 for a full month of sheer stress. I've had to reverse engineer crap design and code both in hopes of making them work as expected.

I had hoped 4.2 would finally get there with performance, but now I'm being shamed in the 3.4 admin panel that I'll stop getting security updates for the version that actually consistently works. Nothing irritates me more than using out of date in software and/or security... So I'm starting to look at alternatives at this point as I get flashbacks to my horrible experience with vB3...

Or more likely just start replacing it all with something of my own...

Link to comment
Share on other sites
RevengeFNF Experienced

RevengeFNF

Posted
Posted
8 minutes ago, ExiledVip3r said:

I have a community of currently ~1,500,000 members, ~1,000,000 daily pageviews, and a 9gb database for the forums.

I really want to upgrade, and I've tried twice now to upgrade my community; first to a midlife 4.0, and again to a midlife 4.1. They both ran like complete and utter crap. When not crashing the servers, they were ideally pushing 25% of the load times I got using 3.4.

Even after optimizing them to the best of my knowledge. Using the same server environment that runs perfectly smooth with 3.4 with power to spare. It has been a complete and utter faeces show with 4.0+ to the point that I've been forced to apologize and revert the community back to the pre-"upgrade" backups both times after trying 4.0 and 4.1 for a full month of sheer stress. I've had to reverse engineer faecesty design and code both in hopes of making them work as expected.

I had hoped 4.2 would finally get there with performance, but now I'm being shamed in the 3.4 admin panel that I'll stop getting security updates for the version that actually consistently works. I'm starting to look at alternatives at this point as I get flashbacks to my horrible experience with vB3...

Can you give us details of the optimization you were using? For example, your tables were in InnoDB? Nginx+php-fpm? etc etc

Link to comment
Share on other sites
ExiledVip3r Explorer

ExiledVip3r

Posted
Posted
35 minutes ago, RevengeFNF said:

Can you give us details of the optimization you were using? For example, your tables were in InnoDB? Nginx+php-fpm? etc etc

Debian 8, 16gb+8cores+ssd, Nginx mainline, MariaDB 10.1 InnoDB, PHP-FPM 5.6/7.0 (tried both, but 7.0 was seemingly only loosely supported at the time), opcache (which was screwed up initially...), and tried pretty much every caching combination possible in the admin panel from nothing, redis, memcached, sql, and file.

 

 

EDIT: Currently running with a dedicated remote SQL server, which wasn't the case during my attempted 4.0-4.1 upgrades. I doubt this would be the deciding factor now though, based off of my previous observations during upgrade, though even it were the ultimate solution, it's a poor one imo.

Link to comment
Share on other sites
inkredible Enthusiast

inkredible

Posted
Posted
On 9.1.2017 at 4:54 AM, Lindy said:

Those having problems with Redis or Memcache, could you please submit a ticket? We are undergoing a comprehensive performance audit and would like to review your configuration. 

Thank you.

I would be happy to see an announcement describing the improvements resulted by this performance audit.

 

It's great to hear such experiences from other fairly large communities. I am running a community with 150k members only, but we do have very large peaks with up to 2k live users (according to analytics) hence I rely on experiences and feedbacks from others in order to optimize the performance of my community as well.

Link to comment
Share on other sites
ProSkill Enthusiast

ProSkill

Posted
Posted

He's my experience for what it's worth. My site gets 15K - 20K visitors per day and I have been having serious slowness issues. Initially my site was running on a VPS with 8GB of RAM and 24 shared cores. I ran into a storage issue where the VPS only allowed up to a 200GB of storage so I had to move to a different server. I first moved to a dedicated server with 4 cores and 8 GB of RAM. I thought being on a VPS would provide better performance, but boy was I wrong. The site quickly overwhelmed the 4 cores and was basically unusable. I then decided my only option was to move back to the VPS, and offload attachments to Amazon S3. Well, Amazon S3 is insanely expensive for bandwidth it was running about 400 per month just for S3 storage, and the site was still slow. So my next try was to move the database over to Amazon RDS which again was insanely expensive it was $11 for about 8 hours of use (imagine the monthly cost) and was very slow due to latency. I then decide to spin up another VPS just for the database, and move the site and attachments back to a dedicated server. Well everything was still very slow and PHP was eating up all the CPU even with no MySQL running on the server, I was getting 64 Server load with 4 cores! So here is the fix and it's like night and day. I installed Easy Apache 4 and installed php 7 with memcached. The site is now faster than it's ever been even with 2k users on. So if your site is slow I would highly recommend upgrading to PHP 7. This took me about 2 months to figure out but, the site is blazing fast now. 

My current setup is: 4 GB 24 core VPS that only runs MySQL, and a 4 core 8 GB dedicated server that runs the forum. 

 

Link to comment
Share on other sites
RevengeFNF Experienced

RevengeFNF

Posted
Posted
6 hours ago, ProSkill said:

He's my experience for what it's worth. My site gets 15K - 20K visitors per day and I have been having serious slowness issues. Initially my site was running on a VPS with 8GB of RAM and 24 shared cores. I ran into a storage issue where the VPS only allowed up to a 200GB of storage so I had to move to a different server. I first moved to a dedicated server with 4 cores and 8 GB of RAM. I thought being on a VPS would provide better performance, but boy was I wrong. The site quickly overwhelmed the 4 cores and was basically unusable. I then decided my only option was to move back to the VPS, and offload attachments to Amazon S3. Well, Amazon S3 is insanely expensive for bandwidth it was running about 400 per month just for S3 storage, and the site was still slow. So my next try was to move the database over to Amazon RDS which again was insanely expensive it was $11 for about 8 hours of use (imagine the monthly cost) and was very slow due to latency. I then decide to spin up another VPS just for the database, and move the site and attachments back to a dedicated server. Well everything was still very slow and PHP was eating up all the CPU even with no MySQL running on the server, I was getting 64 Server load with 4 cores! So here is the fix and it's like night and day. I installed Easy Apache 4 and installed php 7 with memcached. The site is now faster than it's ever been even with 2k users on. So if your site is slow I would highly recommend upgrading to PHP 7. This took me about 2 months to figure out but, the site is blazing fast now. 

My current setup is: 4 GB 24 core VPS that only runs MySQL, and a 4 core 8 GB dedicated server that runs the forum. 

 

You said you were paying 400$ just for the S3. With that kind of money you can buy a Dedicate Server with a lot more than just 4 cores.

For example, for 240$ you can buy a Dedi Server with 20 Cores/40 Threads, 256 GB of Ram and 2x 480Gb SSD.

1 hour ago, marklcfc said:

How did you install memcached with php 7 as I'm always told it is not compatible?

If you use Centos, this one for Remi repo works perfectly: 

Name        : php-pecl-memcached
Arch        : x86_64
Version     : 3.0.2
Release     : 1.el7.remi.7.1
Size        : 267 k
Repo        : installed
From repo   : remi-php71
Summary     : Extension to work with the Memcached caching daemon
URL         : http://pecl.php.net/package/memcached
License     : PHP
Description : This extension uses libmemcached library to provide API for communicating
            : with memcached servers.
            : 
            : memcached is a high-performance, distributed memory object caching system,
            : generic in nature, but intended for use in speeding up dynamic web
            : applications by alleviating database load.
            : 
            : It also provides a session handler (memcached).
            : 
            : Package built for PHP 7.1.

 

Link to comment
Share on other sites
Qubabos Enthusiast

Qubabos

Posted
Posted
On 12.01.2017 at 8:11 PM, SecondSight said:

I also noticed each time I upgraded to a new IPS Community version (for instance, from IPS Community 3.4 to version 4), I had Apache performance problems for some time. I suspect search engine bots to be responsible for this (anyway in the logs I found there were too many connections for bots).

I'm also facing the same problem - 2 different communities, 2 different servers from 2 diferent providers and I get http 429 too many request for googlebot on urls containing string *tags* (hxxp://siteurl/tags/*) - reported 2 times already to IPS and their always saying that is server issue (both run on dualcore xeon with 4 GB of ram, ssd disks, centos and debian and are not heavy loaded with traffic) 

Link to comment
Share on other sites
ProSkill Enthusiast

ProSkill

Posted
Posted
7 hours ago, RevengeFNF said:

You said you were paying 400$ just for the S3. With that kind of money you can buy a Dedicate Server with a lot more than just 4 cores.

For example, for 240$ you can buy a Dedi Server with 20 Cores/40 Threads, 256 GB of Ram and 2x 480Gb SSD.

If you use Centos, this one for Remi repo works perfectly: 


Name        : php-pecl-memcached
Arch        : x86_64
Version     : 3.0.2
Release     : 1.el7.remi.7.1
Size        : 267 k
Repo        : installed
From repo   : remi-php71
Summary     : Extension to work with the Memcached caching daemon
URL         : http://pecl.php.net/package/memcached
License     : PHP
Description : This extension uses libmemcached library to provide API for communicating
            : with memcached servers.
            : 
            : memcached is a high-performance, distributed memory object caching system,
            : generic in nature, but intended for use in speeding up dynamic web
            : applications by alleviating database load.
            : 
            : It also provides a session handler (memcached).
            : 
            : Package built for PHP 7.1.

 

I run an adult site and I don't trust many of the dedicated server hosting companies. A lot of them just don't seem to be run very professionally and they aren't afraid to suspend accounts over little things (one of my accounts got suspended because of a "DDoS attack"  when really it was just a lot of legitimate traffic). Also, they often times don't have the same backup options, security features, or support. So yes a dedicated server with great specs might be cheaper but it's also more risky imho. 

Link to comment
Share on other sites
ASTRAPI Proficient

ASTRAPI

Posted
Posted

I manage also forums with a lot more traffic....At pick times we have 4.000 users online at the same time :-)

More than 20.000.000 posts and database size at 80GB and users produce almost 2 GB of data daily.

Most reports for IPB 4 bad performance are related to use of many addons/apps.

Also it will be an issue if an addon/app is bad coded without performance in mind...

Also other parameters for bad performance is if users don't use any caching like zend opcache or Memcached....

One more reason is that even users use very good software on the server they do not optimize it!

There is a ton of related adjustments as for very high traffic forums users may get limited by the system OS like tcp limitations, open file limitations and so on...

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
Invision Community © 2024 IPS, Inc.
×
×
  • Create New...