当前位置：首页 > news >正文

关于python的Django项目性能优化

news 来源：原创 2024/9/19 12:36:07

CPU——

Python编写的Django程序在服务器上消耗CPU的主要原因包括频繁的IO操作、大量的计算任务、以及第三方库的性能问题。‌

‌频繁的IO操作‌：当Python程序频繁进行IO操作，如读写文件、网络请求等，会导致CPU占用过高。这是因为IO操作通常是相对较慢的，当程序在等待IO操作完成时，CPU会一直处于运行状态，从而导致CPU占用率增加。例如，频繁地读写大文件或进行大量的网络请求都可能导致CPU占用过高。
‌大量的计算任务‌：如果Python程序需要进行大量的计算任务，比如遍历大型数据集、执行复杂的数学运算等，也会导致CPU占用过高。这些计算密集型任务需要CPU进行大量的运算，从而增加了CPU的使用率。
‌第三方库的性能问题‌：使用的第三方库可能存在性能问题，导致在使用过程中占用了过多的CPU资源。在使用第三方库时，需要注意其性能表现，并及时更新到最新版本，以减少对CPU的过度使用。

为了降低Python进程的CPU占用，可以采取以下几种解决方法：

检查代码中的无限循环，通过添加适当的退出条件来解决。
优化IO操作，使用异步IO或多线程技术来处理IO操作，以减少CPU的占用。
及时更新第三方库到最新版本，以解决可能的性能问题。

通过这些措施，可以有效地降低Python Django程序对服务器CPU的消耗，提高服务器的性能和响应速度‌

两次单项测试的cpu性能消耗问题

ncalls  tottime  percall  cumtime  percall filename:lineno(function)196    0.008    0.000    9.509    0.049 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/client.py:768(execute_command)1    0.004    0.004    8.107    8.107 /Users/cengyang/code/media_manager/tools/task_tool.py:849(ban_task)304    0.003    0.000    7.985    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:634(read_response)304    0.010    0.000    7.982    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:289(read_response)304    0.016    0.000    7.971    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:218(readline)304    0.007    0.000    7.952    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:174(_read_from_socket)304    0.001    0.000    7.944    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/_compat.py:57(recv)304    7.943    0.026    7.943    0.026 {method 'recv' of '_socket.socket' objects}36    0.005    0.000    7.612    0.211 /Users/cengyang/code/media_manager/tools/task_tool.py:645(counts_out)55    0.001    0.000    7.047    0.128 /Users/cengyang/code/media_manager/tools/dao_tools.py:203(get)98    0.003    0.000    6.919    0.071 /Users/cengyang/code/media_manager/tools/dao_tools.py:113(get_pool)98    0.001    0.000    6.915    0.071 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/client.py:1035(ping)196    0.003    0.000    5.117    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/client.py:786(parse_response)196    0.005    0.000    4.364    0.022 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:984(get_connection)196    0.005    0.000    4.351    0.022 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:487(connect)1    0.000    0.000    3.251    3.251 /Users/cengyang/code/media_manager/tools/task_tool.py:931(set_child_tasks)11    0.001    0.000    3.249    0.295 /Users/cengyang/code/media_manager/tools/task_tool.py:780(set_task_json)54    0.003    0.000    2.889    0.054 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:563(on_connect)83    0.001    0.000    2.816    0.034 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/backends/utils.py:67(execute)83    0.001    0.000    2.815    0.034 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/backends/utils.py:73(_execute_with_wrappers)83    0.002    0.000    2.815    0.034 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/backends/utils.py:79(_execute)83    0.001    0.000    2.812    0.034 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/backends/mysql/base.py:68(execute)83    0.001    0.000    2.812    0.034 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/cursors.py:151(execute)83    0.002    0.000    2.802    0.034 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/cursors.py:324(_query)83    0.002    0.000    2.798    0.034 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/connections.py:508(query)83    0.001    0.000    2.790    0.034 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/connections.py:720(_read_query_result)83    0.002    0.000    2.789    0.034 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/connections.py:1073(read)3416    0.084    0.000    2.719    0.001 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/connections.py:648(_read_packet)6832    0.012    0.000    2.221    0.000 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/connections.py:687(_read_bytes)6832    0.010    0.000    2.195    0.000 {method 'read' of '_io.BufferedReader' objects}116    0.001    0.000    2.185    0.019 /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py:572(readinto)116    2.183    0.019    2.183    0.019 {method 'recv_into' of '_socket.socket' objects}44    0.003    0.000    2.183    0.050 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/models/query.py:1177(_fetch_all)340    0.005    0.000    2.180    0.006 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/models/query.py:47(__iter__)85/82    0.002    0.000    2.147    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/models/manager.py:81(manager_method)43    0.001    0.000    2.047    0.048 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/models/sql/compiler.py:1039(execute_sql)
38410/13256    0.006    0.000    1.910    0.000 {built-in method builtins.len}41    0.000    0.000    1.907    0.047 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/models/query.py:253(__len__)37    0.001    0.000    1.840    0.050 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/models/query.py:389(get)27    0.001    0.000    1.563    0.058 /Users/cengyang/code/media_manager/tools/dao_tools.py:198(set)55    0.001    0.000    1.459    0.027 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/client.py:1260(get)54    0.004    0.000    1.455    0.027 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:513(_connect)54    1.406    0.026    1.406    0.026 {method 'connect' of '_socket.socket' objects}ncalls  tottime  percall  cumtime  percall filename:lineno(function)300    0.014    0.000   13.850    0.046 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/client.py:768(execute_command)452    0.005    0.000   11.689    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:634(read_response)452    0.017    0.000   11.683    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:289(read_response)452    0.027    0.000   11.664    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:218(readline)452    0.012    0.000   11.633    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:174(_read_from_socket)452    0.001    0.000   11.619    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/_compat.py:57(recv)452   11.617    0.026   11.617    0.026 {method 'recv' of '_socket.socket' objects}150    0.005    0.000    9.941    0.066 /Users/cengyang/code/media_manager/tools/dao_tools.py:113(get_pool)150    0.001    0.000    9.933    0.066 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/client.py:1035(ping)83    0.002    0.000    8.483    0.102 /Users/cengyang/code/media_manager/tools/dao_tools.py:203(get)1    0.004    0.004    8.152    8.152 /Users/cengyang/code/media_manager/tools/task_tool.py:852(ban_task)300    0.005    0.000    7.706    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/client.py:786(parse_response)36    0.006    0.000    7.659    0.213 /Users/cengyang/code/media_manager/tools/task_tool.py:648(counts_out)12    0.003    0.000    6.692    0.558 /Users/cengyang/code/media_manager/tools/task_tool.py:284(task_can_do)300    0.009    0.000    6.092    0.020 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:984(get_connection)300    0.006    0.000    6.068    0.020 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:487(connect)76    0.006    0.000    4.018    0.053 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:563(on_connect)104    0.001    0.000    3.420    0.033 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/backends/utils.py:67(execute)104    0.001    0.000    3.419    0.033 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/backends/utils.py:73(_execute_with_wrappers)104    0.003    0.000    3.418    0.033 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/backends/utils.py:79(_execute)104    0.001    0.000    3.415    0.033 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/backends/mysql/base.py:68(execute)104    0.002    0.000    3.415    0.033 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/cursors.py:151(execute)104    0.002    0.000    3.400    0.033 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/cursors.py:324(_query)104    0.002    0.000    3.396    0.033 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/connections.py:508(query)104    0.001    0.000    3.386    0.033 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/connections.py:720(_read_query_result)104    0.003    0.000    3.384    0.033 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/connections.py:1073(read)3694    0.098    0.000    3.302    0.001 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/connections.py:648(_read_packet)11    0.002    0.000    2.940    0.267 /Users/cengyang/code/media_manager/tools/task_tool.py:600(time_key_cando)65    0.004    0.000    2.871    0.044 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/models/query.py:1177(_fetch_all)367    0.007    0.000    2.867    0.008 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/models/query.py:47(__iter__)7388    0.013    0.000    2.733    0.000 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/pymysql/connections.py:687(_read_bytes)64    0.002    0.000    2.703    0.042 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/models/sql/compiler.py:1039(execute_sql)7388    0.012    0.000    2.703    0.000 {method 'read' of '_io.BufferedReader' objects}32    0.001    0.000    2.701    0.084 /Users/cengyang/code/media_manager/tools/dao_tools.py:198(set)134    0.001    0.000    2.691    0.020 /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py:572(readinto)134    2.688    0.020    2.688    0.020 {method 'recv_into' of '_socket.socket' objects}
42797/15544    0.007    0.000    2.601    0.000 {built-in method builtins.len}62    0.000    0.000    2.597    0.042 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/models/query.py:253(__len__)106/103    0.003    0.000    2.257    0.022 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/django/db/models/manager.py:81(manager_method)83    0.001    0.000    2.172    0.026 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/client.py:1260(get)76    0.004    0.000    2.041    0.027 /Users/cengyang/code/envs/demo2/lib/python3.6/site-packages/redis/connection.py:513(_connect)76    1.968    0.026    1.968    0.026 {method 'connect' of '_socket.socket' objects}

内存——

redis——

RDS——

带宽——